[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:55:34 dist-test-slave-7vjv krb5kdc[4270](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:55:34 dist-test-slave-7vjv krb5kdc[4270](info): set up 2 sockets
Aug 11 06:55:34 dist-test-slave-7vjv krb5kdc[4270](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 06:55:35.474050 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.524s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:55:35 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:55:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:55:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:55:35.670208 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46353
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:46353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:35.997471 4293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:35.997823 4293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:35.998016 4293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:36.005089 4293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:55:36.005245 4293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:36.005358 4293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:36.005475 4293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:55:36.005565 4293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:55:36.015187 4293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:46353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:36.016535 4293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:36.017912 4293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:36.030125 4303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:36.030726 4308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:36.030759 4301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:36.030994 4293 server_base.cc:1034] running on GCE node
I20260811 06:55:36.031867 4293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:36.033380 4293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:36.034624 4293 hybrid_clock.cc:648] HybridClock initialized: now 1786431336034633 us; error 152 us; skew 500 ppm
I20260811 06:55:36.034983 4293 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:36 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:36.038607 4293 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:55:36.040337 4293 webserver.cc:460] Webserver started at http://127.4.40.62:37001/ using document root <none> and password file <none>
I20260811 06:55:36.041023 4293 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:36.041141 4293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:36.041435 4293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:36.044755 4293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "1a2edef3226b4a55b3ce70f85929be93"
format_stamp: "Formatted at 2026-08-11 06:55:36 on dist-test-slave-7vjv"
server_key: "553339767b3b493583b9c2b84806963f"
server_key_iv: "68d3218ca7241526c7e91b0f3593fed5"
server_key_version: "encryptionkey@0"
I20260811 06:55:36.045545 4293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "1a2edef3226b4a55b3ce70f85929be93"
format_stamp: "Formatted at 2026-08-11 06:55:36 on dist-test-slave-7vjv"
server_key: "553339767b3b493583b9c2b84806963f"
server_key_iv: "68d3218ca7241526c7e91b0f3593fed5"
server_key_version: "encryptionkey@0"
I20260811 06:55:36.051069 4293 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:55:36.054652 4327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:36.055939 4293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:55:36.056147 4293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "1a2edef3226b4a55b3ce70f85929be93"
format_stamp: "Formatted at 2026-08-11 06:55:36 on dist-test-slave-7vjv"
server_key: "553339767b3b493583b9c2b84806963f"
server_key_iv: "68d3218ca7241526c7e91b0f3593fed5"
server_key_version: "encryptionkey@0"
I20260811 06:55:36.056356 4293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:36.079505 4293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:36.083727 4293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:36.084024 4293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:36.103400 4293 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:46353
I20260811 06:55:36.103449 4413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:46353 every 8 connection(s)
I20260811 06:55:36.104799 4293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:55:36.108539 4418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:36.109551 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4293
I20260811 06:55:36.109860 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:55:36.111747 4256 external_mini_cluster.cc:1428] Setting key 7f19135c5111631fa993e892622cbc15
I20260811 06:55:36.118999 4418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93: Bootstrap starting.
I20260811 06:55:36.121917 4418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:36.122897 4418 log.cc:824] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:36.125912 4418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93: No bootstrap required, opened a new log
I20260811 06:55:36.135252 4418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2edef3226b4a55b3ce70f85929be93" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46353 } }
I20260811 06:55:36.135707 4418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:36.135818 4418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a2edef3226b4a55b3ce70f85929be93, State: Initialized, Role: FOLLOWER
I20260811 06:55:36.136538 4418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2edef3226b4a55b3ce70f85929be93" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46353 } }
I20260811 06:55:36.136778 4418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:55:36.136946 4418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:55:36.137115 4418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:36.139196 4418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2edef3226b4a55b3ce70f85929be93" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46353 } }
I20260811 06:55:36.139688 4418 leader_election.cc:302] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a2edef3226b4a55b3ce70f85929be93; no voters:
I20260811 06:55:36.140347 4418 leader_election.cc:288] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:55:36.140498 4431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 1 FOLLOWER]: Leader election won for term 1
Aug 11 06:55:36 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:36.141204 4431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [term 1 LEADER]: Becoming Leader. State: Replica: 1a2edef3226b4a55b3ce70f85929be93, State: Running, Role: LEADER
I20260811 06:55:36.141863 4431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2edef3226b4a55b3ce70f85929be93" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46353 } }
I20260811 06:55:36.143113 4418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:55:36.144991 4434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a2edef3226b4a55b3ce70f85929be93. Latest consensus state: current_term: 1 leader_uuid: "1a2edef3226b4a55b3ce70f85929be93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2edef3226b4a55b3ce70f85929be93" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46353 } } }
I20260811 06:55:36.145380 4434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:36.149739 4436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a2edef3226b4a55b3ce70f85929be93" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a2edef3226b4a55b3ce70f85929be93" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46353 } } }
I20260811 06:55:36.150077 4436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93 [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:36.151434 4442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:55:36.157992 4423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:36.115691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56764 (local address 127.4.40.62:46353)
0811 06:55:36.116653 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:36.116671 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:36.117955 (+ 1284us) server_negotiation.cc:407] Connection header received
0811 06:55:36.127661 (+ 9706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:36.127686 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:36.128154 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:36.128589 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:36.131265 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:36.133337 (+ 2072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:36.135041 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:36.136085 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:36.136287 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:36.143036 (+ 6749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:36.143049 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:36.143068 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:36.143118 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:36.148479 (+ 5361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:36.149780 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:36.149790 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:36.149800 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:36.150064 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:36.150769 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:36.150775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:36.150785 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:36.151880 (+ 1095us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:36.155014 (+ 3134us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:36.155409 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 06:55:36.155969 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":438,"threads_started":1}
I20260811 06:55:36.158088 4442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:55:36.169631 4442 catalog_manager.cc:1347] Generated new cluster ID: a7b92a1bd8a34713950f52ae1a1da5c3
I20260811 06:55:36.169785 4442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:55:36.201022 4442 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:55:36.202366 4442 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:55:36.210368 4442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a2edef3226b4a55b3ce70f85929be93: Generated new TSK 0
I20260811 06:55:36.211308 4442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:55:36.271730 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46353
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:36.611603 4470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:36.611930 4470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:36.612146 4470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:36.619073 4470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:36.619236 4470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:36.619432 4470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:55:36.629065 4470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46353
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:36.630350 4470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:36.631737 4470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:36.645413 4482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:36.645861 4487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:36.645936 4484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:36.646257 4470 server_base.cc:1034] running on GCE node
I20260811 06:55:36.646941 4470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:36.647912 4470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:36.649118 4470 hybrid_clock.cc:648] HybridClock initialized: now 1786431336649054 us; error 54 us; skew 500 ppm
I20260811 06:55:36.649442 4470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:36 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:36.652843 4470 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:55:36.654346 4470 webserver.cc:460] Webserver started at http://127.4.40.1:42831/ using document root <none> and password file <none>
I20260811 06:55:36.655056 4470 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:36.655210 4470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:36.655540 4470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:36.658605 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "23b3ec5673fb46c7891794dc75690340"
format_stamp: "Formatted at 2026-08-11 06:55:36 on dist-test-slave-7vjv"
server_key: "a6c179bf6b525f0abbbf6ae671a757cd"
server_key_iv: "931de6359074ecaaee54c26540e7ff65"
server_key_version: "encryptionkey@0"
I20260811 06:55:36.659473 4470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "23b3ec5673fb46c7891794dc75690340"
format_stamp: "Formatted at 2026-08-11 06:55:36 on dist-test-slave-7vjv"
server_key: "a6c179bf6b525f0abbbf6ae671a757cd"
server_key_iv: "931de6359074ecaaee54c26540e7ff65"
server_key_version: "encryptionkey@0"
I20260811 06:55:36.665192 4470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 06:55:36.668797 4498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:36.670039 4470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:55:36.670220 4470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "23b3ec5673fb46c7891794dc75690340"
format_stamp: "Formatted at 2026-08-11 06:55:36 on dist-test-slave-7vjv"
server_key: "a6c179bf6b525f0abbbf6ae671a757cd"
server_key_iv: "931de6359074ecaaee54c26540e7ff65"
server_key_version: "encryptionkey@0"
I20260811 06:55:36.670446 4470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:36.699770 4470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:36.704339 4470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:36.704686 4470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:36.705705 4470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:55:36.707070 4470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:36.707183 4470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:36.707342 4470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:36.707429 4470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:36.742872 4470 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:35367
I20260811 06:55:36.742903 4726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:35367 every 8 connection(s)
I20260811 06:55:36.744444 4470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:55:36.752439 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4470
I20260811 06:55:36.752735 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:55:36.753410 4256 external_mini_cluster.cc:1428] Setting key 8ceb539541787520919540cc5b8d7de7
Aug 11 06:55:36 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:36.767177 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:36.747211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:37761 (local address 127.4.40.62:46353)
0811 06:55:36.747694 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:36.747699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:36.748585 (+ 886us) server_negotiation.cc:407] Connection header received
0811 06:55:36.750446 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:36.750452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:36.750548 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:36.750666 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:36.752340 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:36.753347 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:36.755379 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:36.755954 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:36.756127 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:36.760701 (+ 4574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:36.760710 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:36.760718 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:36.760763 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:36.763851 (+ 3088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:36.764822 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:36.764826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:36.764828 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:36.764953 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:36.765462 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:36.765465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:36.765467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:36.765728 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:36.765858 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:36.766768 (+ 910us) server_negotiation.cc:299] Negotiation successful
0811 06:55:36.766929 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":275,"threads_started":1}
I20260811 06:55:36.768743 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:36.747612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46353 (local address 127.4.40.1:37761)
0811 06:55:36.748411 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:36.748459 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:36.750175 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:36.750915 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:36.750940 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:36.751441 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:36.752163 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:36.752191 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:36.753522 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:36.753529 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:36.755235 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:36.755252 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:36.756230 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:36.756236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:36.756445 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:36.757200 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:36.757231 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:36.760395 (+ 3164us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:36.764003 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:36.764015 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:36.764034 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:36.764703 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:36.765058 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:36.765065 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:36.765072 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:36.765332 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:36.765867 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:36.765877 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:36.766373 (+ 496us) client_negotiation.cc:769] Sending connection context
0811 06:55:36.766675 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 06:55:36.767267 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":292,"threads_started":1}
I20260811 06:55:36.770383 4728 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46353
I20260811 06:55:36.770735 4728 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:36.771692 4728 heartbeater.cc:511] Master 127.4.40.62:46353 requested a full tablet report, sending...
I20260811 06:55:36.774322 4362 ts_manager.cc:194] Registered new tserver with Master: 23b3ec5673fb46c7891794dc75690340 (127.4.40.1:35367)
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
I20260811 06:55:36.776420 4362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:37761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:55:36.803005 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46353
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:37.145848 4737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:37.146112 4737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:37.146209 4737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:37.153435 4737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:37.153535 4737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:37.153676 4737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:55:37.163115 4737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46353
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:37.164456 4737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:37.165794 4737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:37.178695 4744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:37.178656 4745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:37.178794 4748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:37.179893 4737 server_base.cc:1034] running on GCE node
I20260811 06:55:37.180505 4737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:37.181419 4737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:37.182626 4737 hybrid_clock.cc:648] HybridClock initialized: now 1786431337182576 us; error 36 us; skew 500 ppm
I20260811 06:55:37.182960 4737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:37.186003 4737 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:55:37.187414 4737 webserver.cc:460] Webserver started at http://127.4.40.2:46223/ using document root <none> and password file <none>
I20260811 06:55:37.188053 4737 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:37.188290 4737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:37.188650 4737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:37.191485 4737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "b0ce4cfc619e4565bcf8cef4dfe9ac1a"
format_stamp: "Formatted at 2026-08-11 06:55:37 on dist-test-slave-7vjv"
server_key: "09547a7c41b14496483d297a6fd9b6e6"
server_key_iv: "119463d7cc958737fd4fb67275270e3f"
server_key_version: "encryptionkey@0"
I20260811 06:55:37.192319 4737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "b0ce4cfc619e4565bcf8cef4dfe9ac1a"
format_stamp: "Formatted at 2026-08-11 06:55:37 on dist-test-slave-7vjv"
server_key: "09547a7c41b14496483d297a6fd9b6e6"
server_key_iv: "119463d7cc958737fd4fb67275270e3f"
server_key_version: "encryptionkey@0"
I20260811 06:55:37.197496 4737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:55:37.201570 4768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:37.202819 4737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:55:37.202943 4737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "b0ce4cfc619e4565bcf8cef4dfe9ac1a"
format_stamp: "Formatted at 2026-08-11 06:55:37 on dist-test-slave-7vjv"
server_key: "09547a7c41b14496483d297a6fd9b6e6"
server_key_iv: "119463d7cc958737fd4fb67275270e3f"
server_key_version: "encryptionkey@0"
I20260811 06:55:37.203112 4737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:37.221249 4737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:37.225284 4737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:37.225540 4737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:37.226501 4737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:55:37.227846 4737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:37.227919 4737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:37.228025 4737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:37.228070 4737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:37.262774 4737 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:38321
I20260811 06:55:37.262826 4993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:38321 every 8 connection(s)
I20260811 06:55:37.264056 4737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:55:37.273077 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4737
I20260811 06:55:37.273295 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:55:37.273818 4256 external_mini_cluster.cc:1428] Setting key 237e50566b9b6ebc6217035045f39ccc
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:37.285142 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.266568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:45133 (local address 127.4.40.62:46353)
0811 06:55:37.266799 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:37.266805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:37.267902 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 06:55:37.269955 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:37.269959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:37.270027 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:37.270114 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:37.271491 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.272277 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.274395 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.275082 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.275180 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.279320 (+ 4140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:37.279325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:37.279328 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:37.279358 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:37.282031 (+ 2673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.282775 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.282780 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.282785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.282922 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.283375 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.283382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.283390 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.283649 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:37.283812 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:37.284531 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 06:55:37.284772 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:55:37.286077 4999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.267057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46353 (local address 127.4.40.2:45133)
0811 06:55:37.267745 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:37.267789 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:37.269687 (+ 1898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:37.270311 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:37.270319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:37.270798 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:37.271345 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:37.271361 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.272407 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.272411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:37.274261 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:37.274276 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.275317 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.275324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:37.275606 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.276360 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:37.276413 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:37.279098 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:37.282181 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:37.282189 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:37.282203 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:37.282643 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:37.283036 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:37.283042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:37.283045 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:37.283231 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:37.283784 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:37.283792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:37.284169 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 06:55:37.284445 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 06:55:37.284944 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":287,"threads_started":1}
I20260811 06:55:37.287547 4997 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46353
I20260811 06:55:37.287829 4997 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:37.288630 4997 heartbeater.cc:511] Master 127.4.40.62:46353 requested a full tablet report, sending...
I20260811 06:55:37.290207 4362 ts_manager.cc:194] Registered new tserver with Master: b0ce4cfc619e4565bcf8cef4dfe9ac1a (127.4.40.2:38321)
I20260811 06:55:37.291458 4362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:45133
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:55:37.319458 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46353
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:37.650825 5008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:37.651095 5008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:37.651281 5008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:37.658492 5008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:37.658624 5008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:37.658810 5008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:55:37.668109 5008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46353
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:37.669335 5008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:37.670643 5008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:37.684442 5018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:37.684474 5023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:37.684523 5020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:37.685247 5008 server_base.cc:1034] running on GCE node
I20260811 06:55:37.685890 5008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:37.686790 5008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:37.687981 5008 hybrid_clock.cc:648] HybridClock initialized: now 1786431337687924 us; error 48 us; skew 500 ppm
I20260811 06:55:37.688314 5008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:37.691345 5008 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:55:37.692775 5008 webserver.cc:460] Webserver started at http://127.4.40.3:37993/ using document root <none> and password file <none>
I20260811 06:55:37.693398 5008 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:37.693501 5008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:37.693749 5008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:37.696751 5008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "0c0413ff53b047f8961639a021ea2c44"
format_stamp: "Formatted at 2026-08-11 06:55:37 on dist-test-slave-7vjv"
server_key: "90d22fa0faba56e7b9e8a7fcb7c2f39d"
server_key_iv: "a3819ef9b0a50c029742f4882073d7d3"
server_key_version: "encryptionkey@0"
I20260811 06:55:37.697461 5008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "0c0413ff53b047f8961639a021ea2c44"
format_stamp: "Formatted at 2026-08-11 06:55:37 on dist-test-slave-7vjv"
server_key: "90d22fa0faba56e7b9e8a7fcb7c2f39d"
server_key_iv: "a3819ef9b0a50c029742f4882073d7d3"
server_key_version: "encryptionkey@0"
I20260811 06:55:37.702481 5008 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:55:37.705909 5037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:37.707326 5008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:55:37.707458 5008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "0c0413ff53b047f8961639a021ea2c44"
format_stamp: "Formatted at 2026-08-11 06:55:37 on dist-test-slave-7vjv"
server_key: "90d22fa0faba56e7b9e8a7fcb7c2f39d"
server_key_iv: "a3819ef9b0a50c029742f4882073d7d3"
server_key_version: "encryptionkey@0"
I20260811 06:55:37.707628 5008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:37.730114 5008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:37.733934 5008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:37.734174 5008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:37.735085 5008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:55:37.736433 5008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:37.736505 5008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:37.736588 5008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:37.736631 5008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:37.773487 5239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:46421 every 8 connection(s)
I20260811 06:55:37.773365 5008 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:46421
I20260811 06:55:37.775141 5008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:55:37.777062 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5008
I20260811 06:55:37.777282 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:55:37.777927 4256 external_mini_cluster.cc:1428] Setting key baf8058ad0907ccd93c28dd69de8d9b7
I20260811 06:55:37.779605 4728 heartbeater.cc:503] Master 127.4.40.62:46353 was elected leader, sending a full tablet report...
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:37.795401 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.777829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:48159 (local address 127.4.40.62:46353)
0811 06:55:37.778043 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:37.778048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:37.779460 (+ 1412us) server_negotiation.cc:407] Connection header received
0811 06:55:37.781934 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:37.781939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:37.782020 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:37.782124 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:37.783528 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.784355 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.785769 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.786152 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.786243 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.789518 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:37.789522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:37.789526 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:37.789555 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:37.792193 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.792896 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.792901 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.792904 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.793070 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.793543 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.793549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.793554 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.793895 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:37.794057 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:37.794829 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 06:55:37.795078 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:55:37.796305 5249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.778433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46353 (local address 127.4.40.3:48159)
0811 06:55:37.779302 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:37.779350 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:37.781693 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:37.782351 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:37.782358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:37.782835 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:37.783349 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:37.783364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.784495 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.784499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:37.785629 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:37.785659 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.786326 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.786330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:37.786525 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.787155 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:37.787174 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:37.789313 (+ 2139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:37.792314 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:37.792320 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:37.792329 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:37.792722 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:37.793198 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:37.793205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:37.793211 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:37.793413 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:37.794033 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:37.794040 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:37.794398 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 06:55:37.794718 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 06:55:37.795210 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":349,"threads_started":1}
I20260811 06:55:37.797492 5240 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46353
I20260811 06:55:37.797770 5240 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:37.798461 5240 heartbeater.cc:511] Master 127.4.40.62:46353 requested a full tablet report, sending...
I20260811 06:55:37.799750 4362 ts_manager.cc:194] Registered new tserver with Master: 0c0413ff53b047f8961639a021ea2c44 (127.4.40.3:46421)
I20260811 06:55:37.800571 4362 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:48159
I20260811 06:55:37.807049 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:37.858738 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.843281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38264 (local address 127.4.40.1:35367)
0811 06:55:37.844037 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:37.844046 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:37.844079 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:55:37.844282 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:37.844290 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:37.844575 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:37.844795 (+ 220us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:37.846257 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.847298 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.848689 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.849257 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.849402 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.853525 (+ 4123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:37.853536 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:37.853549 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:37.853581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:37.856362 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.856963 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.856971 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.856974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.857096 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.857411 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.857414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.857415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.857744 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:37.857965 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:37.858156 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 06:55:37.858342 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":359,"threads_started":1}
W20260811 06:55:37.860219 4566 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38264
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:55:37 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:37.908631 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.894874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.4.40.1:35367)
0811 06:55:37.895115 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:37.895122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:37.895396 (+ 274us) server_negotiation.cc:407] Connection header received
0811 06:55:37.895557 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:37.895561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:37.895656 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:37.895744 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:37.897161 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.897865 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.899018 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.899412 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.899510 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.903503 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:37.903517 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:37.903520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:37.903551 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:37.906309 (+ 2758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.906917 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.906921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.906923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.907040 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.907438 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.907443 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.907446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.907757 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:37.907953 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:37.908255 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:55:37.908414 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:55:37.910238 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4470
I20260811 06:55:37.923053 5259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:37.911194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38288 (local address 127.4.40.1:35367)
0811 06:55:37.911378 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:37.911386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:37.911724 (+ 338us) server_negotiation.cc:407] Connection header received
0811 06:55:37.911893 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:37.911898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:37.911960 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:37.912064 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:37.913243 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.913981 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.915175 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:37.915620 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:37.915756 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:37.917982 (+ 2226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:37.917986 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:37.917989 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:37.918019 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:37.920486 (+ 2467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.921120 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.921127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.921131 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.921263 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:37.921631 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:37.921636 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:37.921639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:37.921891 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:37.922148 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:37.922458 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 06:55:37.922689 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:55:37.923758 4718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:38.008456 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4470
I20260811 06:55:38.033586 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4737
Aug 11 06:55:38 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:55:38.050239 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:38.034472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.4.40.2:38321)
0811 06:55:38.035076 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:38.035083 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:38.035110 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:55:38.035182 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:38.035188 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:38.035390 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:38.035531 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:38.036755 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:38.037864 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:38.039240 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:38.039807 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:38.039909 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:38.044254 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:38.044266 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:38.044281 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:38.044313 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:38.047298 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:38.047955 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:38.047963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:38.047965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:38.048161 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:38.048568 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:38.048574 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:38.048578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:38.048927 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:38.049261 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:38.049606 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 06:55:38.049887 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":315,"threads_started":1}
I20260811 06:55:38.051499 4976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:38.136677 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4737
I20260811 06:55:38.167405 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5008
Aug 11 06:55:38 dist-test-slave-7vjv krb5kdc[4270](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:55:38.183074 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:38.168531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46624 (local address 127.4.40.3:46421)
0811 06:55:38.169198 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:38.169205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:38.169236 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:55:38.169295 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:38.169302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:38.169503 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:38.169622 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:38.170802 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:38.171836 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:38.173151 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:38.173643 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:38.173735 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:38.177647 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:38.177658 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:38.177670 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:38.177706 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:38.180678 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:38.181355 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:38.181362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:38.181365 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:38.181494 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:38.181810 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:38.181813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:38.181815 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:38.182164 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:38.182415 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:38.182607 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 06:55:38.182785 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":358,"threads_started":1}
I20260811 06:55:38.184314 5234 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:38.267373 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5008
I20260811 06:55:38.294394 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4293
I20260811 06:55:38.295693 4404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:38.404927 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4293
2026-08-11T06:55:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4518 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:55:38 dist-test-slave-7vjv krb5kdc[5284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:55:38 dist-test-slave-7vjv krb5kdc[5284](info): set up 2 sockets
Aug 11 06:55:38 dist-test-slave-7vjv krb5kdc[5284](info): commencing operation
krb5kdc: starting...
W20260811 06:55:39.746273 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:55:39 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:55:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:55:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:55:39.947000 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46227
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:46227
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:40.285591 5306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:40.285938 5306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:40.286125 5306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:40.294478 5306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:55:40.294600 5306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:40.294651 5306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:40.294762 5306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:55:40.294857 5306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:55:40.305078 5306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:46227
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:40.306442 5306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:40.307888 5306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:40.320556 5314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:40.320600 5316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:40.320648 5321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:40.321667 5306 server_base.cc:1034] running on GCE node
I20260811 06:55:40.322475 5306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:40.323954 5306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:40.325245 5306 hybrid_clock.cc:648] HybridClock initialized: now 1786431340325220 us; error 54 us; skew 500 ppm
I20260811 06:55:40.325603 5306 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:40 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:40.332774 5306 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:55:40.334256 5306 webserver.cc:460] Webserver started at http://127.4.40.62:45687/ using document root <none> and password file <none>
I20260811 06:55:40.335007 5306 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:40.335150 5306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:40.335551 5306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:40.339533 5306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "36f3de83d5bc49d7acdd8394d9a96f58"
format_stamp: "Formatted at 2026-08-11 06:55:40 on dist-test-slave-7vjv"
server_key: "ca7aefd1c6202f5c01510ee0c751de31"
server_key_iv: "51c1474a4263235b3d0606b37a70cf57"
server_key_version: "encryptionkey@0"
I20260811 06:55:40.340468 5306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "36f3de83d5bc49d7acdd8394d9a96f58"
format_stamp: "Formatted at 2026-08-11 06:55:40 on dist-test-slave-7vjv"
server_key: "ca7aefd1c6202f5c01510ee0c751de31"
server_key_iv: "51c1474a4263235b3d0606b37a70cf57"
server_key_version: "encryptionkey@0"
I20260811 06:55:40.347527 5306 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 06:55:40.351541 5331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:40.352882 5306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:40.353063 5306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "36f3de83d5bc49d7acdd8394d9a96f58"
format_stamp: "Formatted at 2026-08-11 06:55:40 on dist-test-slave-7vjv"
server_key: "ca7aefd1c6202f5c01510ee0c751de31"
server_key_iv: "51c1474a4263235b3d0606b37a70cf57"
server_key_version: "encryptionkey@0"
I20260811 06:55:40.353257 5306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:40.374823 5306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:40.379246 5306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:40.379555 5306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:40.398391 5306 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:46227
I20260811 06:55:40.398429 5421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:46227 every 8 connection(s)
I20260811 06:55:40.399816 5306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:55:40.403456 5422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:40.404597 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5306
I20260811 06:55:40.404850 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:55:40.405750 4256 external_mini_cluster.cc:1428] Setting key e050c5fbec0a05762b7b24caed7bf41b
I20260811 06:55:40.418181 5422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58: Bootstrap starting.
Aug 11 06:55:40 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:40.421788 5422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:40.422842 5422 log.cc:824] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:40.426048 5422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58: No bootstrap required, opened a new log
I20260811 06:55:40.429855 5428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:40.407821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.4.40.62:46227)
0811 06:55:40.408684 (+ 863us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:40.408695 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:40.408737 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:55:40.410330 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:40.410355 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:40.410727 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:40.411047 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:40.412894 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:40.414149 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:40.415942 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:40.416646 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:40.416796 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:40.421489 (+ 4693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:40.421497 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:40.421517 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:40.421559 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:40.425302 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:40.425943 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:40.425951 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:40.425958 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:40.426191 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:40.426558 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:40.426563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:40.426566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:40.427245 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:40.427592 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:40.427988 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 06:55:40.428496 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":419,"threads_started":1}
I20260811 06:55:40.435432 5422 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46227 } }
I20260811 06:55:40.435791 5422 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:40.435941 5422 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36f3de83d5bc49d7acdd8394d9a96f58, State: Initialized, Role: FOLLOWER
I20260811 06:55:40.436611 5422 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46227 } }
I20260811 06:55:40.436862 5422 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:55:40.437006 5422 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:55:40.437194 5422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:40.439556 5422 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46227 } }
I20260811 06:55:40.440053 5422 leader_election.cc:302] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36f3de83d5bc49d7acdd8394d9a96f58; no voters:
I20260811 06:55:40.440696 5422 leader_election.cc:288] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:55:40.440850 5432 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:40.441618 5432 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [term 1 LEADER]: Becoming Leader. State: Replica: 36f3de83d5bc49d7acdd8394d9a96f58, State: Running, Role: LEADER
I20260811 06:55:40.442137 5432 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46227 } }
I20260811 06:55:40.442842 5422 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:55:40.445468 5433 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46227 } } }
I20260811 06:55:40.445936 5433 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:40.445441 5435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36f3de83d5bc49d7acdd8394d9a96f58. Latest consensus state: current_term: 1 leader_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36f3de83d5bc49d7acdd8394d9a96f58" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46227 } } }
I20260811 06:55:40.446646 5435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58 [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:40.447405 5440 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:55:40.453410 5440 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:55:40.462988 5462 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:55:40.464182 5440 catalog_manager.cc:1347] Generated new cluster ID: 68c051c5705942e2a90e99b578d2574c
I20260811 06:55:40.464288 5440 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:55:40.484172 5440 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:55:40.485996 5440 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:55:40.501849 5440 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36f3de83d5bc49d7acdd8394d9a96f58: Generated new TSK 0
I20260811 06:55:40.502887 5440 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:55:40.549773 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46227
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:40.878228 5472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:40.878471 5472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:40.878563 5472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:40.885695 5472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:40.885797 5472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:40.885905 5472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:55:40.895216 5472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:40.896507 5472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:40.897893 5472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:40.911948 5479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:40.911994 5480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:40.912147 5483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:40.912521 5472 server_base.cc:1034] running on GCE node
I20260811 06:55:40.913264 5472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:40.914176 5472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:40.915378 5472 hybrid_clock.cc:648] HybridClock initialized: now 1786431340915316 us; error 53 us; skew 500 ppm
I20260811 06:55:40.915778 5472 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:40 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:40.918965 5472 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:55:40.920588 5472 webserver.cc:460] Webserver started at http://127.4.40.1:45167/ using document root <none> and password file <none>
I20260811 06:55:40.921347 5472 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:40.921509 5472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:40.921842 5472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:40.924952 5472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "227b1a6cb60645dea1622cad00523e11"
format_stamp: "Formatted at 2026-08-11 06:55:40 on dist-test-slave-7vjv"
server_key: "c17b5de76125a24fd7eb6e7efdff89f2"
server_key_iv: "4784362bdfc6c90a3cf177de46aad51e"
server_key_version: "encryptionkey@0"
I20260811 06:55:40.925806 5472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "227b1a6cb60645dea1622cad00523e11"
format_stamp: "Formatted at 2026-08-11 06:55:40 on dist-test-slave-7vjv"
server_key: "c17b5de76125a24fd7eb6e7efdff89f2"
server_key_iv: "4784362bdfc6c90a3cf177de46aad51e"
server_key_version: "encryptionkey@0"
I20260811 06:55:40.931208 5472 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:55:40.934733 5495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:40.936195 5472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:40.936381 5472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "227b1a6cb60645dea1622cad00523e11"
format_stamp: "Formatted at 2026-08-11 06:55:40 on dist-test-slave-7vjv"
server_key: "c17b5de76125a24fd7eb6e7efdff89f2"
server_key_iv: "4784362bdfc6c90a3cf177de46aad51e"
server_key_version: "encryptionkey@0"
I20260811 06:55:40.936576 5472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:40.969214 5472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:40.974020 5472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:40.974314 5472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:40.975314 5472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:55:40.976807 5472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:40.976913 5472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:40.977074 5472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:40.977159 5472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:41.011945 5472 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:33783
I20260811 06:55:41.011984 5693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:33783 every 8 connection(s)
I20260811 06:55:41.013348 5472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:55:41.017697 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5472
I20260811 06:55:41.017923 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:55:41.018561 4256 external_mini_cluster.cc:1428] Setting key eb5177cd4b0f8865fdc14454d7d5a3d8
Aug 11 06:55:41 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:41.033789 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:41.015861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:44591 (local address 127.4.40.62:46227)
0811 06:55:41.016395 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:41.016399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:41.017297 (+ 898us) server_negotiation.cc:407] Connection header received
0811 06:55:41.019536 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:41.019541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:41.019667 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:41.019797 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:41.021440 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.022178 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.023740 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.024207 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.024358 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:41.027863 (+ 3505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:41.027868 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:41.027871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:41.027903 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:41.030622 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:41.031333 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:41.031342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:41.031346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:41.031489 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:41.031931 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:41.031937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:41.031940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:41.032287 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:41.032444 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:41.033158 (+ 714us) server_negotiation.cc:299] Negotiation successful
0811 06:55:41.033385 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":285,"threads_started":1}
I20260811 06:55:41.034777 5697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:41.016399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46227 (local address 127.4.40.1:44591)
0811 06:55:41.017093 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:41.017156 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:41.019239 (+ 2083us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:41.020046 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:41.020055 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:41.020702 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:41.021260 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:41.021277 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.022340 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.022347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:41.023588 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:41.023601 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.024449 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.024457 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:41.024702 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:41.025381 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:41.025405 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:41.027641 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:41.030787 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:41.030795 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:41.030809 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:41.031207 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:41.031606 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:41.031610 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:41.031614 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:41.031814 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:41.032419 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:41.032427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:41.032763 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 06:55:41.033022 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 06:55:41.033609 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":294,"threads_started":1}
I20260811 06:55:41.036523 5694 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46227
I20260811 06:55:41.036877 5694 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:41.037840 5694 heartbeater.cc:511] Master 127.4.40.62:46227 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
I20260811 06:55:41.040488 5368 ts_manager.cc:194] Registered new tserver with Master: 227b1a6cb60645dea1622cad00523e11 (127.4.40.1:33783)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
I20260811 06:55:41.042306 5368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:44591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:55:41.067780 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46227
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:41.403053 5703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:41.403402 5703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:41.403561 5703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:41.414621 5703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:41.414762 5703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:41.414955 5703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:55:41.426733 5703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:41.427834 5703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:41.429175 5703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:41.442552 5710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:41.442518 5711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:41.442682 5713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:41.443815 5703 server_base.cc:1034] running on GCE node
I20260811 06:55:41.444499 5703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:41.445334 5703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:41.446547 5703 hybrid_clock.cc:648] HybridClock initialized: now 1786431341446472 us; error 58 us; skew 500 ppm
I20260811 06:55:41.446856 5703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:41 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431341, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:41.449944 5703 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:55:41.451376 5703 webserver.cc:460] Webserver started at http://127.4.40.2:34031/ using document root <none> and password file <none>
I20260811 06:55:41.452064 5703 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:41.452263 5703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:41.452582 5703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:41.455437 5703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "bfda0be1cf564e988aacbeda5f88cbf5"
format_stamp: "Formatted at 2026-08-11 06:55:41 on dist-test-slave-7vjv"
server_key: "b2389b59c656b2d07cc21c6f149173c1"
server_key_iv: "b8046c650a7c8e16ed75f0f24ca0788e"
server_key_version: "encryptionkey@0"
I20260811 06:55:41.456317 5703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "bfda0be1cf564e988aacbeda5f88cbf5"
format_stamp: "Formatted at 2026-08-11 06:55:41 on dist-test-slave-7vjv"
server_key: "b2389b59c656b2d07cc21c6f149173c1"
server_key_iv: "b8046c650a7c8e16ed75f0f24ca0788e"
server_key_version: "encryptionkey@0"
I20260811 06:55:41.461300 5703 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 06:55:41.464668 5726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:41.465973 5703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:55:41.466152 5703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "bfda0be1cf564e988aacbeda5f88cbf5"
format_stamp: "Formatted at 2026-08-11 06:55:41 on dist-test-slave-7vjv"
server_key: "b2389b59c656b2d07cc21c6f149173c1"
server_key_iv: "b8046c650a7c8e16ed75f0f24ca0788e"
server_key_version: "encryptionkey@0"
I20260811 06:55:41.466344 5703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:41.489199 5703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:41.493511 5703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:41.493840 5703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:41.494864 5703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:55:41.496284 5703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:41.496411 5703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:41.496572 5703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:41.496652 5703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:41.533401 5703 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:35945
I20260811 06:55:41.533452 5917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:35945 every 8 connection(s)
I20260811 06:55:41.534799 5703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:55:41.537930 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5703
I20260811 06:55:41.538144 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:55:41.538583 4256 external_mini_cluster.cc:1428] Setting key 9812b173ec7c98fa56e836453ebb59eb
Aug 11 06:55:41 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431341, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:41.556485 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:41.537528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:42857 (local address 127.4.40.62:46227)
0811 06:55:41.538062 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:41.538071 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:41.539256 (+ 1185us) server_negotiation.cc:407] Connection header received
0811 06:55:41.541614 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:41.541620 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:41.541705 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:41.541829 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:41.543365 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.544525 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.546128 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.546886 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.547024 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:41.550629 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:41.550634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:41.550637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:41.550668 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:41.553347 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:41.554078 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:41.554083 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:41.554086 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:41.554231 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:41.554659 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:41.554664 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:41.554667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:41.554937 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:41.555097 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:41.555780 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 06:55:41.556076 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":261,"threads_started":1}
I20260811 06:55:41.557438 5924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:41.538135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46227 (local address 127.4.40.2:42857)
0811 06:55:41.539077 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:41.539130 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:41.541318 (+ 2188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:41.541994 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:41.542002 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:41.542537 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:41.543185 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:41.543214 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.544702 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.544706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:41.545969 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:41.545983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:41.547100 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:41.547106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:41.547297 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:41.548021 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:41.548043 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:41.550404 (+ 2361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:41.553503 (+ 3099us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:41.553522 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:41.553536 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:41.553922 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:41.554350 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:41.554355 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:41.554359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:41.554543 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:41.555067 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:41.555075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:41.555392 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 06:55:41.555674 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:55:41.556220 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":392,"threads_started":1}
I20260811 06:55:41.558835 5918 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46227
I20260811 06:55:41.559170 5918 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
I20260811 06:55:41.560083 5918 heartbeater.cc:511] Master 127.4.40.62:46227 requested a full tablet report, sending...
I20260811 06:55:41.561640 5368 ts_manager.cc:194] Registered new tserver with Master: bfda0be1cf564e988aacbeda5f88cbf5 (127.4.40.2:35945)
I20260811 06:55:41.562474 5368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:42857
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:55:41.586925 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46227
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:32987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:41.911859 5930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:41.912133 5930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:41.912233 5930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:41.919152 5930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:41.919274 5930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:41.919386 5930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:55:41.928709 5930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:32987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46227
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:41.929843 5930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:41.931087 5930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:41.944253 5945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:41.944271 5943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:41.944646 5941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:41.945449 5930 server_base.cc:1034] running on GCE node
I20260811 06:55:41.946125 5930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:41.946971 5930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:41.948216 5930 hybrid_clock.cc:648] HybridClock initialized: now 1786431341948149 us; error 51 us; skew 500 ppm
I20260811 06:55:41.948495 5930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:41 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431341, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:41.951498 5930 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:55:41.953048 5930 webserver.cc:460] Webserver started at http://127.4.40.3:33023/ using document root <none> and password file <none>
I20260811 06:55:41.953774 5930 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:41.953919 5930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:41.954231 5930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:41.957199 5930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c"
format_stamp: "Formatted at 2026-08-11 06:55:41 on dist-test-slave-7vjv"
server_key: "a3160419e4bfc4469653dd5526d45481"
server_key_iv: "60c75f9c7f3b4c472c6602dff7492ecf"
server_key_version: "encryptionkey@0"
I20260811 06:55:41.958073 5930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c"
format_stamp: "Formatted at 2026-08-11 06:55:41 on dist-test-slave-7vjv"
server_key: "a3160419e4bfc4469653dd5526d45481"
server_key_iv: "60c75f9c7f3b4c472c6602dff7492ecf"
server_key_version: "encryptionkey@0"
I20260811 06:55:41.963124 5930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:55:41.967730 5962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:41.969092 5930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:41.969328 5930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c"
format_stamp: "Formatted at 2026-08-11 06:55:41 on dist-test-slave-7vjv"
server_key: "a3160419e4bfc4469653dd5526d45481"
server_key_iv: "60c75f9c7f3b4c472c6602dff7492ecf"
server_key_version: "encryptionkey@0"
I20260811 06:55:41.969543 5930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:41.986943 5930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:41.991266 5930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:41.991510 5930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:41.992491 5930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:55:41.993757 5930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:41.993831 5930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:41.993916 5930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:41.993963 5930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:42.027657 5930 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:41601
I20260811 06:55:42.027707 6160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:41601 every 8 connection(s)
I20260811 06:55:42.029004 5930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:55:42.036054 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5930
I20260811 06:55:42.036301 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:55:42.036830 4256 external_mini_cluster.cc:1428] Setting key 893c2e33ce95ee6cbc79f77f0cfe7eab
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431341, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:42.045477 5694 heartbeater.cc:503] Master 127.4.40.62:46227 was elected leader, sending a full tablet report...
I20260811 06:55:42.048869 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.031592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:44681 (local address 127.4.40.62:46227)
0811 06:55:42.031793 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.031797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.032868 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 06:55:42.034736 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.034743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.034850 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.034965 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.036618 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.037598 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.039107 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.039544 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.039695 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.043133 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.043138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.043141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.043174 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.045851 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.046552 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.046562 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.046564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.046687 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.047123 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.047128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.047135 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.047385 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.047551 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.048317 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.048543 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:55:42.049774 6170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.031900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46227 (local address 127.4.40.3:44681)
0811 06:55:42.032698 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:42.032746 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:42.034446 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:42.035204 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:42.035212 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:42.035730 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:42.036419 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.036450 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.037749 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.037754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.038954 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.038968 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.039783 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.039789 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.040024 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.040724 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:42.040745 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:42.042915 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:42.045999 (+ 3084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:42.046007 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:42.046021 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:42.046414 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:42.046812 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:42.046817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:42.046820 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:42.047002 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:42.047522 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:42.047530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:42.047879 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 06:55:42.048166 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 06:55:42.048657 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":347,"threads_started":1}
I20260811 06:55:42.051190 6162 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46227
I20260811 06:55:42.051538 6162 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:42.052387 6162 heartbeater.cc:511] Master 127.4.40.62:46227 requested a full tablet report, sending...
I20260811 06:55:42.053857 5367 ts_manager.cc:194] Registered new tserver with Master: cf71a9b588ad4aa8b4bc0d0bba88910c (127.4.40.3:41601)
I20260811 06:55:42.054653 5367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:44681
I20260811 06:55:42.064208 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:55:42.082722 5920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.070405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.4.40.62:46227)
0811 06:55:42.070624 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.070632 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.071190 (+ 558us) server_negotiation.cc:407] Connection header received
0811 06:55:42.071314 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.071318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.071378 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.071460 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.073135 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.073906 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.075129 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.075538 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.075674 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.077933 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.077937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.077940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.077968 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.080412 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.080998 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.081001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.081003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.081122 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.081432 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.081435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.081441 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.081724 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.081914 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.082170 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.082353 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:55:42.088596 5367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:42.095279 5367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:42.123817 6194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.107546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:41601 (local address 127.0.0.1:50326)
0811 06:55:42.108287 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:42.108313 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:42.108430 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:42.109917 (+ 1487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:42.109929 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:42.109970 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:42.110537 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.110554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.113496 (+ 2942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.113505 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.116659 (+ 3154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.116682 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.119859 (+ 3177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.119869 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.120262 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.120353 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 06:55:42.120492 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:55:42.120749 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"thread_start_us":1110,"threads_started":1}
I20260811 06:55:42.125490 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.108328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50326 (local address 127.4.40.3:41601)
0811 06:55:42.109142 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.109153 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.109196 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:55:42.109321 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.109333 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.109617 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.110803 (+ 1186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:42.111157 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.113290 (+ 2133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.116993 (+ 3703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.119560 (+ 2567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.124560 (+ 5000us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.124821 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.124920 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.125107 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":502,"threads_started":1}
I20260811 06:55:42.130316 6042 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9a52b8974a4183a458a3b733a02f0f (DEFAULT_TABLE table=test-table [id=50bca55ca55b48bba395da3bc8732e60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:42.132345 6042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9a52b8974a4183a458a3b733a02f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:42.133586 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.108389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33783 (local address 127.0.0.1:43770)
0811 06:55:42.121385 (+ 12996us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:42.121410 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:42.121571 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:42.124859 (+ 3288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:42.124865 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:42.124891 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:42.126293 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.126305 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.128316 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.128322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.130439 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.130454 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.132627 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.132634 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.132834 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.132875 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 06:55:42.132964 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 06:55:42.133101 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12806,"mutex_wait_us":1,"thread_start_us":13283,"threads_started":1}
I20260811 06:55:42.134857 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.108851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.4.40.1:33783)
0811 06:55:42.123550 (+ 14699us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.123561 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.123824 (+ 263us) server_negotiation.cc:407] Connection header received
0811 06:55:42.124026 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.124036 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.124348 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.124566 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:42.126523 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.128130 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.130640 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.132281 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.133933 (+ 1652us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.134209 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.134263 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.134500 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14493,"thread_start_us":14306,"threads_started":1}
I20260811 06:55:42.137188 6187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.106813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35945 (local address 127.0.0.1:54918)
0811 06:55:42.107920 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:42.107976 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:42.108171 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:42.127141 (+ 18970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:42.127146 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:42.127171 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:42.127508 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.127519 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.130735 (+ 3216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.130740 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.133893 (+ 3153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.133908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.136328 (+ 2420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.136336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.136502 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.136534 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:55:42.136645 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:55:42.136778 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":1049,"thread_start_us":723,"threads_started":1}
I20260811 06:55:42.137297 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.106995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54918 (local address 127.4.40.2:35945)
0811 06:55:42.126539 (+ 19544us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.126548 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.126575 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:55:42.126673 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.126681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.126909 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.127698 (+ 789us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:42.127917 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.130586 (+ 2669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.134132 (+ 3546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.135560 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.135699 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.135878 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.136721 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.136915 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19365,"thread_start_us":25735,"threads_started":1}
I20260811 06:55:42.138808 5582 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9a52b8974a4183a458a3b733a02f0f (DEFAULT_TABLE table=test-table [id=50bca55ca55b48bba395da3bc8732e60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:42.140537 5582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9a52b8974a4183a458a3b733a02f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:42.146492 5811 tablet_service.cc:1467] Processing CreateTablet for tablet 2c9a52b8974a4183a458a3b733a02f0f (DEFAULT_TABLE table=test-table [id=50bca55ca55b48bba395da3bc8732e60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:42.147722 5811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c9a52b8974a4183a458a3b733a02f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:42.152853 6200 tablet_bootstrap.cc:492] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: Bootstrap starting.
I20260811 06:55:42.156705 6201 tablet_bootstrap.cc:492] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: Bootstrap starting.
I20260811 06:55:42.156744 6200 tablet_bootstrap.cc:654] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:42.158274 6200 log.cc:824] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:42.160646 6201 tablet_bootstrap.cc:654] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:42.162117 6200 tablet_bootstrap.cc:492] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: No bootstrap required, opened a new log
I20260811 06:55:42.162387 6200 ts_tablet_manager.cc:1397] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 06:55:42.163064 6201 log.cc:824] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:42.166565 6201 tablet_bootstrap.cc:492] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: No bootstrap required, opened a new log
I20260811 06:55:42.166942 6201 ts_tablet_manager.cc:1397] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 06:55:42.167773 6205 tablet_bootstrap.cc:492] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: Bootstrap starting.
I20260811 06:55:42.171538 6205 tablet_bootstrap.cc:654] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:42.172806 6205 log.cc:824] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:42.176587 6200 raft_consensus.cc:348] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.177269 6200 raft_consensus.cc:374] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:42.177419 6200 raft_consensus.cc:729] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf71a9b588ad4aa8b4bc0d0bba88910c, State: Initialized, Role: FOLLOWER
I20260811 06:55:42.178200 6200 consensus_queue.cc:260] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.179529 6162 heartbeater.cc:503] Master 127.4.40.62:46227 was elected leader, sending a full tablet report...
I20260811 06:55:42.179898 6201 raft_consensus.cc:348] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.180403 6201 raft_consensus.cc:374] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:42.180424 6200 ts_tablet_manager.cc:1428] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: Time spent starting tablet: real 0.018s user 0.018s sys 0.000s
I20260811 06:55:42.180567 6201 raft_consensus.cc:729] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227b1a6cb60645dea1622cad00523e11, State: Initialized, Role: FOLLOWER
I20260811 06:55:42.181231 6201 consensus_queue.cc:260] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.182971 6201 ts_tablet_manager.cc:1428] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260811 06:55:42.184264 6205 tablet_bootstrap.cc:492] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: No bootstrap required, opened a new log
I20260811 06:55:42.184697 6205 ts_tablet_manager.cc:1397] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.008s
I20260811 06:55:42.195329 6205 raft_consensus.cc:348] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.195736 6205 raft_consensus.cc:374] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:42.195878 6205 raft_consensus.cc:729] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfda0be1cf564e988aacbeda5f88cbf5, State: Initialized, Role: FOLLOWER
I20260811 06:55:42.196524 6205 consensus_queue.cc:260] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.197702 5918 heartbeater.cc:503] Master 127.4.40.62:46227 was elected leader, sending a full tablet report...
I20260811 06:55:42.198264 6205 ts_tablet_manager.cc:1428] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260811 06:55:42.231700 6211 raft_consensus.cc:482] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:42.232069 6211 raft_consensus.cc:504] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.233876 6211 leader_election.cc:288] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfda0be1cf564e988aacbeda5f88cbf5 (127.4.40.2:35945), cf71a9b588ad4aa8b4bc0d0bba88910c (127.4.40.3:41601)
I20260811 06:55:42.242949 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.234426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:55273 (local address 127.4.40.3:41601)
0811 06:55:42.234631 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.234639 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.235541 (+ 902us) server_negotiation.cc:407] Connection header received
0811 06:55:42.235774 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.235779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.235849 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.235942 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:42.236743 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.237802 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.240898 (+ 3096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.242010 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.242138 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.242257 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.242526 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.242662 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:55:42.243043 6215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.234345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:41601 (local address 127.4.40.1:55273)
0811 06:55:42.235386 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:42.235408 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:42.235635 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:42.235991 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:42.235995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:42.236030 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:42.236543 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.236573 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.237960 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.237968 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.240739 (+ 2771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.240753 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.242215 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.242219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.242344 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.242370 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:55:42.242455 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 06:55:42.242548 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":928,"mutex_wait_us":169,"spinlock_wait_cycles":35328,"thread_start_us":498,"threads_started":1}
I20260811 06:55:42.243739 6216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.234282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35945 (local address 127.4.40.1:35043)
0811 06:55:42.235386 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:42.235420 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:42.235577 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:42.235991 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:42.235995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:42.236035 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:42.236617 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.236626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.237978 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.237982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.240737 (+ 2755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:42.240753 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.242950 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.242956 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:42.243143 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.243181 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:55:42.243269 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:55:42.243387 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":944,"mutex_wait_us":162,"spinlock_wait_cycles":4352,"thread_start_us":652,"threads_started":1}
I20260811 06:55:42.243763 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.234498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:35043 (local address 127.4.40.2:35945)
0811 06:55:42.234694 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.234701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.235547 (+ 846us) server_negotiation.cc:407] Connection header received
0811 06:55:42.235770 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.235775 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.235849 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.235942 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:42.236743 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.237843 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.241414 (+ 3571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.242637 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.242776 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.242864 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.243435 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.243588 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:55:42.244189 6077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9a52b8974a4183a458a3b733a02f0f" candidate_uuid: "227b1a6cb60645dea1622cad00523e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" is_pre_election: true
I20260811 06:55:42.244802 6077 raft_consensus.cc:2454] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227b1a6cb60645dea1622cad00523e11 in term 0.
I20260811 06:55:42.244930 5844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9a52b8974a4183a458a3b733a02f0f" candidate_uuid: "227b1a6cb60645dea1622cad00523e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" is_pre_election: true
I20260811 06:55:42.245499 5844 raft_consensus.cc:2454] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 227b1a6cb60645dea1622cad00523e11 in term 0.
I20260811 06:55:42.245574 5507 leader_election.cc:302] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227b1a6cb60645dea1622cad00523e11, cf71a9b588ad4aa8b4bc0d0bba88910c; no voters:
I20260811 06:55:42.246057 6211 raft_consensus.cc:2781] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:42.246234 6211 raft_consensus.cc:482] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:42.246388 6211 raft_consensus.cc:3037] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:42.249049 6211 raft_consensus.cc:504] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.249591 6211 leader_election.cc:288] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [CANDIDATE]: Term 1 election: Requested vote from peers bfda0be1cf564e988aacbeda5f88cbf5 (127.4.40.2:35945), cf71a9b588ad4aa8b4bc0d0bba88910c (127.4.40.3:41601)
I20260811 06:55:42.250059 6077 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9a52b8974a4183a458a3b733a02f0f" candidate_uuid: "227b1a6cb60645dea1622cad00523e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c"
I20260811 06:55:42.250288 6077 raft_consensus.cc:3037] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:42.250260 5844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c9a52b8974a4183a458a3b733a02f0f" candidate_uuid: "227b1a6cb60645dea1622cad00523e11" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfda0be1cf564e988aacbeda5f88cbf5"
I20260811 06:55:42.250474 5844 raft_consensus.cc:3037] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:42.252799 6077 raft_consensus.cc:2454] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227b1a6cb60645dea1622cad00523e11 in term 1.
I20260811 06:55:42.252983 5844 raft_consensus.cc:2454] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 227b1a6cb60645dea1622cad00523e11 in term 1.
I20260811 06:55:42.253283 5507 leader_election.cc:302] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 227b1a6cb60645dea1622cad00523e11, cf71a9b588ad4aa8b4bc0d0bba88910c; no voters:
I20260811 06:55:42.253618 6211 raft_consensus.cc:2781] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:42.254030 6211 raft_consensus.cc:686] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [term 1 LEADER]: Becoming Leader. State: Replica: 227b1a6cb60645dea1622cad00523e11, State: Running, Role: LEADER
I20260811 06:55:42.254496 6211 consensus_queue.cc:237] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } }
I20260811 06:55:42.259552 5366 catalog_manager.cc:5495] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11 reported cstate change: term changed from 0 to 1, leader changed from <none> to 227b1a6cb60645dea1622cad00523e11 (127.4.40.1). New cstate: current_term: 1 leader_uuid: "227b1a6cb60645dea1622cad00523e11" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfda0be1cf564e988aacbeda5f88cbf5" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "227b1a6cb60645dea1622cad00523e11" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf71a9b588ad4aa8b4bc0d0bba88910c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41601 } health_report { overall_health: UNKNOWN } } }
W20260811 06:55:42.265723 5695 tablet.cc:2369] T 2c9a52b8974a4183a458a3b733a02f0f P 227b1a6cb60645dea1622cad00523e11: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:55:42.280567 6164 tablet.cc:2369] T 2c9a52b8974a4183a458a3b733a02f0f P cf71a9b588ad4aa8b4bc0d0bba88910c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:55:42.286979 5919 tablet.cc:2369] T 2c9a52b8974a4183a458a3b733a02f0f P bfda0be1cf564e988aacbeda5f88cbf5: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:42.294203 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.278748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.4.40.1:33783)
0811 06:55:42.278924 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.278928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.279442 (+ 514us) server_negotiation.cc:407] Connection header received
0811 06:55:42.279672 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.279676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.279739 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.279843 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.281535 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.282250 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.283525 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.283922 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.284061 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.288418 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.288430 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.288444 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.288479 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.291527 (+ 3048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.292323 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.292330 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.292332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.292486 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.292860 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.292864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.292866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.293215 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.293449 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.293758 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.293962 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431342, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431342, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:42.351542 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.334935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43796 (local address 127.4.40.1:33783)
0811 06:55:42.335197 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.335206 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.335908 (+ 702us) server_negotiation.cc:407] Connection header received
0811 06:55:42.336137 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.336144 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.336269 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.336431 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.338531 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.339865 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.341903 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.342463 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.342569 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.346650 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.346655 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.346658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.346691 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.349384 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.349947 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.349951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.349953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.350093 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.350440 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.350443 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.350445 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.350696 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.350923 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.351110 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.351285 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:42.403991 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.389576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43800 (local address 127.4.40.1:33783)
0811 06:55:42.389805 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.389814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.390193 (+ 379us) server_negotiation.cc:407] Connection header received
0811 06:55:42.390418 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.390423 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.390505 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.390621 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.392050 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.392893 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.394154 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.394597 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.394734 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.398691 (+ 3957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.398696 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.398699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.398733 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.401369 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.402034 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.402041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.402049 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.402184 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.402564 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.402569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.402574 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.402839 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.403130 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.403404 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.403641 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 06:55:42.406257 5548 tablet_service.cc:2908] Rejecting scan request for tablet 2c9a52b8974a4183a458a3b733a02f0f: Uninitialized: clean time has not yet been initialized
I20260811 06:55:42.409641 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5472
I20260811 06:55:42.423781 6199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.410551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43812 (local address 127.4.40.1:33783)
0811 06:55:42.410708 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.410712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.412521 (+ 1809us) server_negotiation.cc:407] Connection header received
0811 06:55:42.412723 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.412727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.412785 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.412889 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.414090 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.414806 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.416027 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.416505 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.416642 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.418774 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.418779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.418782 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.418812 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.421172 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.421822 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.421829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.421832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.421964 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.422349 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.422354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.422357 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.422624 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.422882 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.423185 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.423443 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:55:42.424595 5687 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:42.522789 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5472
I20260811 06:55:42.562299 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5703
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:55:42.577366 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.563238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54924 (local address 127.4.40.2:35945)
0811 06:55:42.563455 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.563462 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.563483 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:55:42.563551 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.563557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.563629 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.563731 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.564962 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.565745 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.567051 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.567443 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.567542 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.571789 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.571801 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.571815 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.571847 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.574653 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.575298 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.575306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.575308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.575436 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.575776 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.575779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.575781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.576336 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.576605 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.576808 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.576992 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:55:42.578096 5913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:42.668109 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5703
I20260811 06:55:42.696969 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5930
Aug 11 06:55:42 dist-test-slave-7vjv krb5kdc[5284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:55:42.712355 6198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:42.697777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50338 (local address 127.4.40.3:41601)
0811 06:55:42.697983 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:42.697991 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:42.698014 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:55:42.698238 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:42.698243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:42.698309 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:42.698418 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:42.699709 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.700517 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.701827 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:42.702238 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:42.702379 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:42.706689 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:42.706704 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:42.706720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:42.706763 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:42.709531 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.710206 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.710214 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.710216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.710354 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:42.710750 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:42.710756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:42.710759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:42.711131 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:42.711436 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:42.711748 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 06:55:42.711978 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:55:42.713038 6153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:42.802953 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5930
I20260811 06:55:42.832613 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5306
I20260811 06:55:42.833945 5414 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:42.945185 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5306
2026-08-11T06:55:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4545 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:55:43 dist-test-slave-7vjv krb5kdc[6251](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:55:43 dist-test-slave-7vjv krb5kdc[6251](info): set up 2 sockets
Aug 11 06:55:43 dist-test-slave-7vjv krb5kdc[6251](info): commencing operation
krb5kdc: starting...
W20260811 06:55:44.336663 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.007s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:55:44 dist-test-slave-7vjv krb5kdc[6251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:55:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:55:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:55:44.550278 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37461
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:37461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:44.880988 6275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:44.881285 6275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:44.881474 6275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:44.888674 6275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:55:44.888773 6275 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:55:44.888895 6275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:44.889008 6275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:44.889093 6275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:55:44.889180 6275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:55:44.898897 6275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:37461
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:44.900267 6275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:44.901618 6275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:44.913729 6286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:44.913764 6288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:44.913733 6292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:44.914973 6275 server_base.cc:1034] running on GCE node
I20260811 06:55:44.915838 6275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:44.917341 6275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:44.918566 6275 hybrid_clock.cc:648] HybridClock initialized: now 1786431344918501 us; error 41 us; skew 500 ppm
I20260811 06:55:44.918913 6275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:44 dist-test-slave-7vjv krb5kdc[6251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:44.922200 6275 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:55:44.923740 6275 webserver.cc:460] Webserver started at http://127.4.40.62:44059/ using document root <none> and password file <none>
I20260811 06:55:44.924520 6275 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:44.924669 6275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:44.925021 6275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:44.927999 6275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "b6d2df16f12d46d7867b85871614acbe"
format_stamp: "Formatted at 2026-08-11 06:55:44 on dist-test-slave-7vjv"
server_key: "3d7a0e66deb06c89e402d486f1ae8ead"
server_key_iv: "ab60c98223bb5151d31962978ebd3aa0"
server_key_version: "encryptionkey@0"
I20260811 06:55:44.928880 6275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "b6d2df16f12d46d7867b85871614acbe"
format_stamp: "Formatted at 2026-08-11 06:55:44 on dist-test-slave-7vjv"
server_key: "3d7a0e66deb06c89e402d486f1ae8ead"
server_key_iv: "ab60c98223bb5151d31962978ebd3aa0"
server_key_version: "encryptionkey@0"
I20260811 06:55:44.933888 6275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:55:44.937336 6302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:44.938609 6275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 06:55:44.938799 6275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "b6d2df16f12d46d7867b85871614acbe"
format_stamp: "Formatted at 2026-08-11 06:55:44 on dist-test-slave-7vjv"
server_key: "3d7a0e66deb06c89e402d486f1ae8ead"
server_key_iv: "ab60c98223bb5151d31962978ebd3aa0"
server_key_version: "encryptionkey@0"
I20260811 06:55:44.939019 6275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:44.964612 6275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:44.969323 6275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:44.969625 6275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:44.993995 6275 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:37461
I20260811 06:55:44.994015 6421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:37461 every 8 connection(s)
I20260811 06:55:44.995443 6275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:55:44.999169 6422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:44.999742 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6275
I20260811 06:55:44.999980 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:55:45.000710 4256 external_mini_cluster.cc:1428] Setting key 1750244cf49a46a3ce28feacdb84a487
I20260811 06:55:45.013432 6422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: Bootstrap starting.
Aug 11 06:55:45 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431344, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:45.017158 6422 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:45.018736 6422 log.cc:824] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:45.021879 6422 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: No bootstrap required, opened a new log
I20260811 06:55:45.024407 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:45.002921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34534 (local address 127.4.40.62:37461)
0811 06:55:45.003883 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:45.003896 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:45.003937 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:55:45.006295 (+ 2358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:45.006317 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:45.006708 (+ 391us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:45.007118 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:45.008696 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.009983 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.011225 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.011902 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.012137 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:45.016289 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:45.016297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:45.016309 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:45.016346 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:45.020242 (+ 3896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:45.020951 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:45.020957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:45.020963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:45.021145 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:45.021548 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:45.021553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:45.021557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:45.022164 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:45.022477 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:45.022906 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 06:55:45.023331 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":741,"thread_start_us":441,"threads_started":1}
I20260811 06:55:45.030443 6422 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d2df16f12d46d7867b85871614acbe" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37461 } }
I20260811 06:55:45.030949 6422 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:45.031078 6422 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d2df16f12d46d7867b85871614acbe, State: Initialized, Role: FOLLOWER
I20260811 06:55:45.031742 6422 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d2df16f12d46d7867b85871614acbe" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37461 } }
I20260811 06:55:45.032081 6422 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:55:45.032259 6422 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:55:45.032455 6422 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:45.036171 6422 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d2df16f12d46d7867b85871614acbe" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37461 } }
I20260811 06:55:45.036849 6422 leader_election.cc:302] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6d2df16f12d46d7867b85871614acbe; no voters:
I20260811 06:55:45.037607 6422 leader_election.cc:288] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:55:45.037838 6431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:45.038581 6431 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [term 1 LEADER]: Becoming Leader. State: Replica: b6d2df16f12d46d7867b85871614acbe, State: Running, Role: LEADER
I20260811 06:55:45.039142 6431 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d2df16f12d46d7867b85871614acbe" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37461 } }
I20260811 06:55:45.040045 6422 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:55:45.043145 6438 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6d2df16f12d46d7867b85871614acbe. Latest consensus state: current_term: 1 leader_uuid: "b6d2df16f12d46d7867b85871614acbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d2df16f12d46d7867b85871614acbe" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37461 } } }
I20260811 06:55:45.043555 6438 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:45.043051 6432 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6d2df16f12d46d7867b85871614acbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6d2df16f12d46d7867b85871614acbe" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37461 } } }
I20260811 06:55:45.044708 6432 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:45.044838 6446 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:55:45.049968 6446 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:55:45.059055 6446 catalog_manager.cc:1347] Generated new cluster ID: 8d306415b34246d598b218a15441df0b
I20260811 06:55:45.059172 6446 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:55:45.083578 6446 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:55:45.084954 6446 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:55:45.091545 6446 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: Generated new TSK 0
I20260811 06:55:45.092605 6446 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:55:45.151916 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37461
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:45.498312 6477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:45.498607 6477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:45.498796 6477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:45.506145 6477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:45.506302 6477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:45.506497 6477 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:55:45.506631 6477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:55:45.517372 6477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:37461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:45.518738 6477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:45.520205 6477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:45.533777 6486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:45.533977 6485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:45.534001 6490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:45.534722 6477 server_base.cc:1034] running on GCE node
I20260811 06:55:45.535312 6477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:45.536216 6477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:45.537473 6477 hybrid_clock.cc:648] HybridClock initialized: now 1786431345537419 us; error 45 us; skew 500 ppm
I20260811 06:55:45.537760 6477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:45 dist-test-slave-7vjv krb5kdc[6251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:45.544737 6477 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:55:45.546257 6477 webserver.cc:460] Webserver started at http://127.4.40.1:35187/ using document root <none> and password file <none>
I20260811 06:55:45.546974 6477 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:45.547104 6477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:45.547441 6477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:45.550437 6477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "088141433f5043e996d0b41085ebb007"
format_stamp: "Formatted at 2026-08-11 06:55:45 on dist-test-slave-7vjv"
server_key: "62b22cbb49d5a0458efe97287598f3e1"
server_key_iv: "9b569ccdb6ba3a67d3f891ceeacf456e"
server_key_version: "encryptionkey@0"
I20260811 06:55:45.551234 6477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "088141433f5043e996d0b41085ebb007"
format_stamp: "Formatted at 2026-08-11 06:55:45 on dist-test-slave-7vjv"
server_key: "62b22cbb49d5a0458efe97287598f3e1"
server_key_iv: "9b569ccdb6ba3a67d3f891ceeacf456e"
server_key_version: "encryptionkey@0"
I20260811 06:55:45.556280 6477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:55:45.559682 6504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:45.560994 6477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:45.561167 6477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "088141433f5043e996d0b41085ebb007"
format_stamp: "Formatted at 2026-08-11 06:55:45 on dist-test-slave-7vjv"
server_key: "62b22cbb49d5a0458efe97287598f3e1"
server_key_iv: "9b569ccdb6ba3a67d3f891ceeacf456e"
server_key_version: "encryptionkey@0"
I20260811 06:55:45.561393 6477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:45.591276 6477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:45.595917 6477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:45.596246 6477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:45.599148 6477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:45.599254 6477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:45.599401 6477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:45.599478 6477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:55:45 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:45.628876 6518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:45.600523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.4.40.1:39901)
0811 06:55:45.601656 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:45.601721 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:45.606208 (+ 4487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:45.607331 (+ 1123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:45.607338 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:45.608294 (+ 956us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:45.609154 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:45.609178 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.610602 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.610609 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:45.613475 (+ 2866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:45.613500 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.614808 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.614814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:45.615086 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:45.615896 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:45.615934 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:45.619109 (+ 3175us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:45.622869 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:45.622876 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:45.622886 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:45.623564 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:45.624068 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:45.624076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:45.624081 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:45.624460 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:45.625141 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:45.625150 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:45.625710 (+ 560us) client_negotiation.cc:769] Sending connection context
0811 06:55:45.626103 (+ 393us) client_negotiation.cc:240] Negotiation successful
0811 06:55:45.626792 (+ 689us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":442,"threads_started":1}
I20260811 06:55:45.629077 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:45.599811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:39901 (local address 127.4.40.62:37461)
0811 06:55:45.602313 (+ 2502us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:45.602320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:45.602348 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:55:45.606605 (+ 4257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:45.606611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:45.606699 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:45.606809 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:45.609427 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.610388 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.613677 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.614310 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.614467 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:45.619333 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:45.619342 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:45.619348 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:45.619395 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:45.622682 (+ 3287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:45.623723 (+ 1041us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:45.623727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:45.623729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:45.623916 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:45.624629 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:45.624634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:45.624640 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:45.624996 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:45.625165 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:45.628350 (+ 3185us) server_negotiation.cc:299] Negotiation successful
0811 06:55:45.628622 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2364,"thread_start_us":2256,"threads_started":1}
I20260811 06:55:45.645187 6477 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:43001
I20260811 06:55:45.645208 6711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:43001 every 8 connection(s)
I20260811 06:55:45.646452 6477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:55:45.648711 6712 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37461
I20260811 06:55:45.649021 6712 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:45.649909 6712 heartbeater.cc:511] Master 127.4.40.62:37461 requested a full tablet report, sending...
I20260811 06:55:45.651566 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:45.635965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:37633 (local address 127.4.40.62:37461)
0811 06:55:45.636233 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:45.636242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:45.637138 (+ 896us) server_negotiation.cc:407] Connection header received
0811 06:55:45.637362 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:45.637369 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:45.637454 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:45.637570 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:45.639022 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.640037 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.642083 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.642888 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.643042 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:45.645177 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:45.645183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:45.645188 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:45.645229 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:45.648437 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:45.649279 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:45.649283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:45.649286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:45.649465 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:45.649914 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:45.649917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:45.649919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:45.650288 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:45.650423 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:45.650886 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 06:55:45.651141 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:55:45.651609 6666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:45.636057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.4.40.1:37633)
0811 06:55:45.637000 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:45.637023 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:45.637193 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:45.637706 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:45.637709 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:45.638384 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:45.638868 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:45.638883 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.640225 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.640236 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:45.641896 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:45.641919 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:45.643096 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:45.643105 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:45.643288 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:45.643987 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:45.644004 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:45.644989 (+ 985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:45.648578 (+ 3589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:45.648583 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:45.648587 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:45.649161 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:45.649605 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:45.649610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:45.649613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:45.649770 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:45.650424 (+ 654us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:45.650430 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:45.650636 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 06:55:45.650786 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:55:45.651112 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":488,"threads_started":1}
I20260811 06:55:45.652361 6340 ts_manager.cc:194] Registered new tserver with Master: 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001)
I20260811 06:55:45.653673 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6477
I20260811 06:55:45.653827 6340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:39901
I20260811 06:55:45.653913 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:55:45.654811 4256 external_mini_cluster.cc:1428] Setting key 4898069163ff8a6fa4d4bd025fb2d9cb
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:55:45.701674 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37461
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:46.033174 6717 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:46.033465 6717 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:46.033563 6717 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:46.040612 6717 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:46.040711 6717 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:46.040804 6717 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:55:46.040892 6717 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:55:46.050729 6717 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:37461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:46.051988 6717 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:46.053390 6717 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:46.066938 6732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:46.067019 6726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:46.067014 6727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:46.067873 6717 server_base.cc:1034] running on GCE node
I20260811 06:55:46.068578 6717 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:46.069473 6717 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:46.070732 6717 hybrid_clock.cc:648] HybridClock initialized: now 1786431346070667 us; error 43 us; skew 500 ppm
I20260811 06:55:46.071039 6717 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:46 dist-test-slave-7vjv krb5kdc[6251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:46.074298 6717 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:55:46.075675 6717 webserver.cc:460] Webserver started at http://127.4.40.2:43289/ using document root <none> and password file <none>
I20260811 06:55:46.076444 6717 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:46.076589 6717 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:46.076928 6717 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:46.079854 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "87c4705025444296945a6a9c015dc5c3"
format_stamp: "Formatted at 2026-08-11 06:55:46 on dist-test-slave-7vjv"
server_key: "ab4fbcc4f26ed69173997ac7df0b01d8"
server_key_iv: "0b47007d207eee3bdf983910b166e2e1"
server_key_version: "encryptionkey@0"
I20260811 06:55:46.080739 6717 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "87c4705025444296945a6a9c015dc5c3"
format_stamp: "Formatted at 2026-08-11 06:55:46 on dist-test-slave-7vjv"
server_key: "ab4fbcc4f26ed69173997ac7df0b01d8"
server_key_iv: "0b47007d207eee3bdf983910b166e2e1"
server_key_version: "encryptionkey@0"
I20260811 06:55:46.085772 6717 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:55:46.089126 6745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:46.090289 6717 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:55:46.090466 6717 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "87c4705025444296945a6a9c015dc5c3"
format_stamp: "Formatted at 2026-08-11 06:55:46 on dist-test-slave-7vjv"
server_key: "ab4fbcc4f26ed69173997ac7df0b01d8"
server_key_iv: "0b47007d207eee3bdf983910b166e2e1"
server_key_version: "encryptionkey@0"
I20260811 06:55:46.090708 6717 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:46.113027 6717 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:46.116894 6717 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:46.117172 6717 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:46.119778 6717 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:46.119895 6717 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:46.120056 6717 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:46.120157 6717 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 06:55:46 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:46.146049 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.120469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:50887 (local address 127.4.40.62:37461)
0811 06:55:46.120742 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.120749 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.122169 (+ 1420us) server_negotiation.cc:407] Connection header received
0811 06:55:46.124653 (+ 2484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.124658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.124758 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.124870 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:46.127565 (+ 2695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.128585 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.131837 (+ 3252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.132542 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.132696 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.138162 (+ 5466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:46.138166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:46.138170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:46.138205 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:46.142112 (+ 3907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.143078 (+ 966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.143084 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.143089 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.143246 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.143942 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.143947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.143950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.144425 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:46.144602 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.145447 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.145682 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148}
I20260811 06:55:46.147184 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.121059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.4.40.2:50887)
0811 06:55:46.121977 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.122022 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.124275 (+ 2253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.125263 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.125277 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.126254 (+ 977us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:46.127313 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.127351 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.128822 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.128827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.131659 (+ 2832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.131682 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.132850 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.132865 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.133235 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.134130 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:46.134216 (+ 86us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:46.137868 (+ 3652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:46.142261 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.142270 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.142291 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.142926 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.143382 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.143387 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.143391 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.143772 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.144678 (+ 906us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:46.144684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:46.145051 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.145358 (+ 307us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.145877 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":416,"threads_started":1}
I20260811 06:55:46.172228 6717 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:43173
I20260811 06:55:46.172271 6956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:43173 every 8 connection(s)
I20260811 06:55:46.173550 6717 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:55:46.175032 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.158898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:39801 (local address 127.4.40.62:37461)
0811 06:55:46.159104 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.159110 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.159884 (+ 774us) server_negotiation.cc:407] Connection header received
0811 06:55:46.160118 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.160123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.160223 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.160332 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:46.162122 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.162856 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.165417 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.166022 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.166192 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.168816 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:46.168824 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:46.168828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:46.168871 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:46.172047 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.172896 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.172903 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.172906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.173035 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.173424 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.173428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.173431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.173829 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:46.173982 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.174436 (+ 454us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.174667 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:55:46.175303 6903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.159072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.4.40.2:39801)
0811 06:55:46.159775 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.159790 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.159964 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.160385 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.160390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.161268 (+ 878us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:46.161906 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.161928 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.163016 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.163022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.165255 (+ 2233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.165273 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.166281 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.166289 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.166541 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.167331 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:46.167363 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:46.168559 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:46.172204 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.172214 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.172218 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.172755 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.173161 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.173166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.173170 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.173305 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.173947 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:46.173952 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:46.174153 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.174332 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.174667 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":424,"threads_started":1}
I20260811 06:55:46.176640 6960 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37461
I20260811 06:55:46.177206 6960 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:46.178016 6960 heartbeater.cc:511] Master 127.4.40.62:37461 requested a full tablet report, sending...
I20260811 06:55:46.179728 6340 ts_manager.cc:194] Registered new tserver with Master: 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173)
I20260811 06:55:46.180647 6340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:50887
I20260811 06:55:46.181783 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6717
I20260811 06:55:46.182006 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:55:46.182440 4256 external_mini_cluster.cc:1428] Setting key 816596eed844fcbb59b350edf5212bf2
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:55:46.231549 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37461
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:46.566993 6967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:46.567329 6967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:46.567497 6967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:46.574668 6967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:46.574810 6967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:46.574965 6967 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:55:46.575114 6967 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:55:46.584417 6967 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:37461
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:46.585695 6967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:46.587023 6967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:46.600973 6975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:46.601048 6979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:46.601097 6976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:46.601874 6967 server_base.cc:1034] running on GCE node
I20260811 06:55:46.602439 6967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:46.603271 6967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:46.604483 6967 hybrid_clock.cc:648] HybridClock initialized: now 1786431346604421 us; error 37 us; skew 500 ppm
I20260811 06:55:46.604739 6967 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:46 dist-test-slave-7vjv krb5kdc[6251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:46.607826 6967 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:55:46.609344 6967 webserver.cc:460] Webserver started at http://127.4.40.3:34493/ using document root <none> and password file <none>
I20260811 06:55:46.610035 6967 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:46.610152 6967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:46.610513 6967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:46.613442 6967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd"
format_stamp: "Formatted at 2026-08-11 06:55:46 on dist-test-slave-7vjv"
server_key: "8c38ffd264d93ac6d2ac826bed22539f"
server_key_iv: "37a7d609078c8578c768762c150caa7e"
server_key_version: "encryptionkey@0"
I20260811 06:55:46.614256 6967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd"
format_stamp: "Formatted at 2026-08-11 06:55:46 on dist-test-slave-7vjv"
server_key: "8c38ffd264d93ac6d2ac826bed22539f"
server_key_iv: "37a7d609078c8578c768762c150caa7e"
server_key_version: "encryptionkey@0"
I20260811 06:55:46.619324 6967 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:55:46.622807 6989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:46.624071 6967 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:55:46.624331 6967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd"
format_stamp: "Formatted at 2026-08-11 06:55:46 on dist-test-slave-7vjv"
server_key: "8c38ffd264d93ac6d2ac826bed22539f"
server_key_iv: "37a7d609078c8578c768762c150caa7e"
server_key_version: "encryptionkey@0"
I20260811 06:55:46.624550 6967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:46.644590 6967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:46.649191 6967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:46.649473 6967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:46.652304 6967 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:46.652412 6967 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:46.652588 6967 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:46.652690 6967 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:46.656952 6712 heartbeater.cc:503] Master 127.4.40.62:37461 was elected leader, sending a full tablet report...
Aug 11 06:55:46 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:46.679729 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.652865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:44881 (local address 127.4.40.62:37461)
0811 06:55:46.653129 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.653135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.654567 (+ 1432us) server_negotiation.cc:407] Connection header received
0811 06:55:46.657691 (+ 3124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.657701 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.657821 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.657909 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:46.660881 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.661822 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.664439 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.665112 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.665253 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.671035 (+ 5782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:46.671046 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:46.671054 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:46.671110 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:46.675425 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.676687 (+ 1262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.676697 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.676703 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.676879 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.677468 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.677473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.677476 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.677918 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:46.678074 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.679067 (+ 993us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.679308 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 06:55:46.680969 7007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.653433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.4.40.3:44881)
0811 06:55:46.654370 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.654422 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.657316 (+ 2894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.658552 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.658562 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.659455 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:46.660639 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.660675 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.662003 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.662012 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.664230 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.664250 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.665363 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.665370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.665731 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.666668 (+ 937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:46.666695 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:46.670671 (+ 3976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:46.675618 (+ 4947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.675631 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.675654 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.676491 (+ 837us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.677046 (+ 555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.677052 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.677061 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.677334 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.678070 (+ 736us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:46.678078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:46.678585 (+ 507us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.678968 (+ 383us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.679524 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":383,"threads_started":1}
I20260811 06:55:46.702486 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.686738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:43527 (local address 127.4.40.62:37461)
0811 06:55:46.686935 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.686943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.687790 (+ 847us) server_negotiation.cc:407] Connection header received
0811 06:55:46.688012 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.688018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.688148 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.688246 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:46.689781 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.690476 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.693138 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.693815 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.693949 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.696337 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:46.696342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:46.696347 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:46.696388 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:46.699040 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.699714 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.699721 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.699724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.699865 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.700316 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.700322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.700326 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.700703 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:46.700863 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.701922 (+ 1059us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.702194 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:55:46.702539 7137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.686813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.4.40.3:43527)
0811 06:55:46.687684 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.687700 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.687866 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.688326 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.688332 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.689109 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:46.689603 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.689620 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.690629 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.690637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.692939 (+ 2302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.692979 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.694172 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.694179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.694287 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.694969 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:46.694990 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:46.696141 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:46.699177 (+ 3036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.699182 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.699186 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.699593 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.699981 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:46.699986 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:46.699990 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:46.700191 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:46.700925 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:46.700936 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:46.701245 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.701463 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.701848 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":492,"threads_started":1}
I20260811 06:55:46.712141 6967 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:39927
I20260811 06:55:46.712234 7218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:39927 every 8 connection(s)
I20260811 06:55:46.713407 6967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:55:46.714222 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6967
I20260811 06:55:46.714458 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:55:46.714995 4256 external_mini_cluster.cc:1428] Setting key a612d5f84ef310ecf886a841c70879b5
I20260811 06:55:46.715866 7220 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37461
I20260811 06:55:46.716176 7220 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:46.716979 7220 heartbeater.cc:511] Master 127.4.40.62:37461 requested a full tablet report, sending...
I20260811 06:55:46.718432 6340 ts_manager.cc:194] Registered new tserver with Master: 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3:39927)
I20260811 06:55:46.719208 6340 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:44881
I20260811 06:55:46.730290 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:55:46 dist-test-slave-7vjv krb5kdc[6251](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:55:46 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:46.783319 6516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.770224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.4.40.62:37461)
0811 06:55:46.770395 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.770399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.770601 (+ 202us) server_negotiation.cc:407] Connection header received
0811 06:55:46.770751 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.770755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.770819 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.770908 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:46.772358 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.773082 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.774352 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.774747 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.774836 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.778539 (+ 3703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:46.778543 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:46.778546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:46.778575 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:46.781088 (+ 2513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.781632 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.781635 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.781637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.781772 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:46.782104 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:46.782107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:46.782109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:46.782348 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:46.782596 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.782793 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.783018 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:55:46.787438 6340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:46.794669 6340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:46.818849 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.806476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33266 (local address 127.4.40.2:43173)
0811 06:55:46.807039 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.807050 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.808058 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 06:55:46.808602 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.808614 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.808888 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.809068 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:46.810081 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.811722 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.815125 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.816968 (+ 1843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.817104 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.817394 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.818276 (+ 882us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.818513 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":326,"threads_started":1}
I20260811 06:55:46.820233 7241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.806224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:43173 (local address 127.0.0.1:33266)
0811 06:55:46.807873 (+ 1649us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.807921 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.808293 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.809404 (+ 1111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.809411 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.809455 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:46.809853 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.809866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.811941 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.811948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.814935 (+ 2987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.814949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.817644 (+ 2695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.817652 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.817936 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.818000 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.819384 (+ 1384us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.819690 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1313,"thread_start_us":920,"threads_started":1}
I20260811 06:55:46.820752 7242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.806440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:39927 (local address 127.0.0.1:45988)
0811 06:55:46.809119 (+ 2679us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.809143 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.809283 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.810769 (+ 1486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.810777 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.810814 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:46.811391 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.811407 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.814464 (+ 3057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.814473 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.817399 (+ 2926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.817420 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.819954 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.819960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.820178 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.820208 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.820275 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.820416 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2500,"thread_start_us":2321,"threads_started":1}
I20260811 06:55:46.821349 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.807139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.4.40.3:39927)
0811 06:55:46.810048 (+ 2909us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.810060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.810100 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:55:46.810215 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.810225 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.810507 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.811670 (+ 1163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:46.812130 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.814260 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.817675 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.819699 (+ 2024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.820426 (+ 727us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.820685 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.820765 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.820952 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2689,"thread_start_us":2525,"threads_started":1}
I20260811 06:55:46.824568 7073 tablet_service.cc:1467] Processing CreateTablet for tablet 5771e939958248d79578fb33d2055692 (DEFAULT_TABLE table=test-table [id=53d47b8556274035b511240deeb0642c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:46.824572 6840 tablet_service.cc:1467] Processing CreateTablet for tablet 5771e939958248d79578fb33d2055692 (DEFAULT_TABLE table=test-table [id=53d47b8556274035b511240deeb0642c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:46.825838 6840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5771e939958248d79578fb33d2055692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:46.825975 7073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5771e939958248d79578fb33d2055692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:46.830489 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.809216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:43001 (local address 127.0.0.1:32838)
0811 06:55:46.819960 (+ 10744us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.819980 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.820252 (+ 272us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.822147 (+ 1895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.822152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.822183 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:46.822531 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.822543 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.825392 (+ 2849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.825397 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.827483 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:46.827500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.829680 (+ 2180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.829690 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:46.829879 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.829914 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:55:46.830018 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:55:46.830153 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10593,"thread_start_us":10896,"threads_started":1}
I20260811 06:55:46.831260 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.809749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32838 (local address 127.4.40.1:43001)
0811 06:55:46.821529 (+ 11780us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.821538 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.821568 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:55:46.821682 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.821692 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.821911 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.822150 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:46.823741 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.825234 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.827672 (+ 2438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:46.829169 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:46.829325 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:46.829528 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:46.830793 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0811 06:55:46.830956 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11605,"thread_start_us":11447,"threads_started":1}
I20260811 06:55:46.835053 6589 tablet_service.cc:1467] Processing CreateTablet for tablet 5771e939958248d79578fb33d2055692 (DEFAULT_TABLE table=test-table [id=53d47b8556274035b511240deeb0642c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:46.837702 6589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5771e939958248d79578fb33d2055692. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:46.838366 7252 tablet_bootstrap.cc:492] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Bootstrap starting.
I20260811 06:55:46.840065 7253 tablet_bootstrap.cc:492] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Bootstrap starting.
I20260811 06:55:46.842320 7252 tablet_bootstrap.cc:654] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:46.843686 7252 log.cc:824] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:46.843742 7253 tablet_bootstrap.cc:654] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:46.844853 7253 log.cc:824] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:46.847167 7252 tablet_bootstrap.cc:492] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: No bootstrap required, opened a new log
I20260811 06:55:46.847509 7252 ts_tablet_manager.cc:1397] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:55:46.849951 7253 tablet_bootstrap.cc:492] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: No bootstrap required, opened a new log
I20260811 06:55:46.850216 7253 ts_tablet_manager.cc:1397] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.005s
I20260811 06:55:46.851095 7257 tablet_bootstrap.cc:492] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Bootstrap starting.
I20260811 06:55:46.853952 7257 tablet_bootstrap.cc:654] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:46.854925 7257 log.cc:824] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:46.857833 7257 tablet_bootstrap.cc:492] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: No bootstrap required, opened a new log
I20260811 06:55:46.858108 7257 ts_tablet_manager.cc:1397] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 06:55:46.858580 7252 raft_consensus.cc:348] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.859004 7252 raft_consensus.cc:374] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:46.859144 7252 raft_consensus.cc:729] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c4705025444296945a6a9c015dc5c3, State: Initialized, Role: FOLLOWER
I20260811 06:55:46.859818 7252 consensus_queue.cc:260] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.861083 7253 raft_consensus.cc:348] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.861605 7253 raft_consensus.cc:374] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:46.861723 7253 raft_consensus.cc:729] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d5e9ec5e6e411d853a7ea47e56e6dd, State: Initialized, Role: FOLLOWER
I20260811 06:55:46.862545 7253 consensus_queue.cc:260] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.863788 7220 heartbeater.cc:503] Master 127.4.40.62:37461 was elected leader, sending a full tablet report...
I20260811 06:55:46.864703 7253 ts_tablet_manager.cc:1428] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 06:55:46.866997 6960 heartbeater.cc:503] Master 127.4.40.62:37461 was elected leader, sending a full tablet report...
I20260811 06:55:46.866989 7257 raft_consensus.cc:348] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.867406 7257 raft_consensus.cc:374] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:46.867527 7257 raft_consensus.cc:729] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088141433f5043e996d0b41085ebb007, State: Initialized, Role: FOLLOWER
I20260811 06:55:46.867937 7252 ts_tablet_manager.cc:1428] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Time spent starting tablet: real 0.020s user 0.012s sys 0.009s
I20260811 06:55:46.868167 7257 consensus_queue.cc:260] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.871592 7257 ts_tablet_manager.cc:1428] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260811 06:55:46.899278 6714 tablet.cc:2369] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:55:46.927673 6962 tablet.cc:2369] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:55:46.965188 7222 tablet.cc:2369] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:55:46.994853 7262 raft_consensus.cc:482] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:46.995188 7262 raft_consensus.cc:504] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:46.997344 7262 leader_election.cc:288] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173), 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3:39927)
I20260811 06:55:47.006217 7247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.997871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:47605 (local address 127.4.40.3:39927)
0811 06:55:46.998106 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.998116 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.999390 (+ 1274us) server_negotiation.cc:407] Connection header received
0811 06:55:46.999552 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.999557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.999645 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.999730 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:47.000553 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.001286 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.004446 (+ 3160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.005240 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.005400 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.005517 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.005806 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.005942 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:55:47.006682 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.997761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:39927 (local address 127.4.40.1:47605)
0811 06:55:46.999289 (+ 1528us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.999304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.999412 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.999881 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.999885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.999908 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:47.000392 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.000407 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.001430 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.001436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.004245 (+ 2809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.004261 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.005457 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.005473 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.005640 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.005666 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.005777 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.005906 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1416,"mutex_wait_us":164,"thread_start_us":868,"threads_started":1}
I20260811 06:55:47.006923 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.997880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:34739 (local address 127.4.40.2:43173)
0811 06:55:46.998344 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:46.998352 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:46.999124 (+ 772us) server_negotiation.cc:407] Connection header received
0811 06:55:46.999502 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:46.999508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:46.999576 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:46.999688 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:47.000371 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.001488 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.004436 (+ 2948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.005668 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.005797 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.005901 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.006476 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.006629 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260811 06:55:47.007117 7265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:46.997561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:43173 (local address 127.4.40.1:34739)
0811 06:55:46.999007 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:46.999031 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:46.999271 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:46.999706 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:46.999710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:46.999736 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:47.000207 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.000223 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.001627 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.001637 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.004246 (+ 2609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.004261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.006079 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.006085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.006265 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.006301 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.006437 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.006725 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1273,"mutex_wait_us":98,"thread_start_us":1448,"threads_started":1}
I20260811 06:55:47.008157 7118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5771e939958248d79578fb33d2055692" candidate_uuid: "088141433f5043e996d0b41085ebb007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" is_pre_election: true
I20260811 06:55:47.008289 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5771e939958248d79578fb33d2055692" candidate_uuid: "088141433f5043e996d0b41085ebb007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c4705025444296945a6a9c015dc5c3" is_pre_election: true
I20260811 06:55:47.008824 6872 raft_consensus.cc:2454] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 088141433f5043e996d0b41085ebb007 in term 0.
I20260811 06:55:47.008824 7118 raft_consensus.cc:2454] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 088141433f5043e996d0b41085ebb007 in term 0.
I20260811 06:55:47.009534 6509 leader_election.cc:302] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d5e9ec5e6e411d853a7ea47e56e6dd, 088141433f5043e996d0b41085ebb007; no voters:
I20260811 06:55:47.009935 7262 raft_consensus.cc:2781] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:47.010107 7262 raft_consensus.cc:482] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:47.010233 7262 raft_consensus.cc:3037] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.012583 7262 raft_consensus.cc:504] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:47.013236 7262 leader_election.cc:288] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [CANDIDATE]: Term 1 election: Requested vote from peers 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173), 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3:39927)
I20260811 06:55:47.013695 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5771e939958248d79578fb33d2055692" candidate_uuid: "088141433f5043e996d0b41085ebb007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c4705025444296945a6a9c015dc5c3"
I20260811 06:55:47.013728 7118 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5771e939958248d79578fb33d2055692" candidate_uuid: "088141433f5043e996d0b41085ebb007" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd"
I20260811 06:55:47.013897 6872 raft_consensus.cc:3037] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.013938 7118 raft_consensus.cc:3037] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.016212 6872 raft_consensus.cc:2454] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 088141433f5043e996d0b41085ebb007 in term 1.
I20260811 06:55:47.016220 7118 raft_consensus.cc:2454] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 088141433f5043e996d0b41085ebb007 in term 1.
I20260811 06:55:47.016662 6505 leader_election.cc:302] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 088141433f5043e996d0b41085ebb007, 87c4705025444296945a6a9c015dc5c3; no voters:
I20260811 06:55:47.017033 7262 raft_consensus.cc:2781] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:47.017472 7262 raft_consensus.cc:686] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 1 LEADER]: Becoming Leader. State: Replica: 088141433f5043e996d0b41085ebb007, State: Running, Role: LEADER
I20260811 06:55:47.017949 7262 consensus_queue.cc:237] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } }
I20260811 06:55:47.022763 6338 catalog_manager.cc:5495] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 reported cstate change: term changed from 0 to 1, leader changed from <none> to 088141433f5043e996d0b41085ebb007 (127.4.40.1). New cstate: current_term: 1 leader_uuid: "088141433f5043e996d0b41085ebb007" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } health_report { overall_health: HEALTHY } } }
I20260811 06:55:47.060451 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.055088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.4.40.1:43001)
0811 06:55:47.055285 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.055289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.055438 (+ 149us) server_negotiation.cc:407] Connection header received
0811 06:55:47.055704 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.055707 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.055804 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.055924 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:47.056322 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.057023 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.058270 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.058654 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.058741 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.059220 (+ 479us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:47.059873 (+ 653us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:47.059979 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.060167 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.060233 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:55:47.068955 7118 raft_consensus.cc:1261] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Refusing update from remote peer 088141433f5043e996d0b41085ebb007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:47.068960 6872 raft_consensus.cc:1261] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Refusing update from remote peer 088141433f5043e996d0b41085ebb007: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:47.069940 7266 consensus_queue.cc:1035] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:47.070324 7262 consensus_queue.cc:1035] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:47.081815 7271 mvcc.cc:205] Tried to move back new op lower bound from 7317222797584080896 to 7317222797388931072. Current Snapshot: MvccSnapshot[applied={T|T < 7317222797584080896}]
I20260811 06:55:47.103685 6340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:47.104007 6340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:47.107786 6340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: Sending DeleteTablet for 3 replicas of tablet 5771e939958248d79578fb33d2055692
I20260811 06:55:47.108981 6840 tablet_service.cc:1514] Processing DeleteTablet for tablet 5771e939958248d79578fb33d2055692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:47 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:33266
I20260811 06:55:47.109041 6589 tablet_service.cc:1514] Processing DeleteTablet for tablet 5771e939958248d79578fb33d2055692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:47 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:32838
I20260811 06:55:47.109994 7073 tablet_service.cc:1514] Processing DeleteTablet for tablet 5771e939958248d79578fb33d2055692 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:47 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:45988
I20260811 06:55:47.110492 6340 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34546:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:47.111269 6340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:47.113015 7284 tablet_replica.cc:330] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: stopping tablet replica
I20260811 06:55:47.113440 7284 raft_consensus.cc:2229] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:55:47.114084 7284 raft_consensus.cc:2258] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:47.115499 7283 tablet_replica.cc:330] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: stopping tablet replica
I20260811 06:55:47.115938 7283 raft_consensus.cc:2229] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:47.116487 7283 raft_consensus.cc:2258] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:47.118505 7284 ts_tablet_manager.cc:1905] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:47.119045 7283 ts_tablet_manager.cc:1905] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:47.120882 6589 tablet_service.cc:1467] Processing CreateTablet for tablet f90a38c5c1b14a61be23108fd4ce10fb (DEFAULT_TABLE table=test-table [id=ff30364197aa40738a9aba068f8eeca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:47.121031 7072 tablet_service.cc:1467] Processing CreateTablet for tablet f90a38c5c1b14a61be23108fd4ce10fb (DEFAULT_TABLE table=test-table [id=ff30364197aa40738a9aba068f8eeca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:47.121346 6589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90a38c5c1b14a61be23108fd4ce10fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:47.121583 7072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90a38c5c1b14a61be23108fd4ce10fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:47.122658 7285 tablet_replica.cc:330] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: stopping tablet replica
I20260811 06:55:47.123078 7285 raft_consensus.cc:2229] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:47.123642 7285 raft_consensus.cc:2258] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:47.126097 7283 ts_tablet_manager.cc:1918] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:55:47.126250 7283 log.cc:1196] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/5771e939958248d79578fb33d2055692
I20260811 06:55:47.126859 7283 ts_tablet_manager.cc:1939] T 5771e939958248d79578fb33d2055692 P 87c4705025444296945a6a9c015dc5c3: Deleting consensus metadata
I20260811 06:55:47.128607 7253 tablet_bootstrap.cc:492] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Bootstrap starting.
I20260811 06:55:47.128847 6840 tablet_service.cc:1467] Processing CreateTablet for tablet f90a38c5c1b14a61be23108fd4ce10fb (DEFAULT_TABLE table=test-table [id=ff30364197aa40738a9aba068f8eeca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:47.129271 6840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f90a38c5c1b14a61be23108fd4ce10fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:47.130306 7253 tablet_bootstrap.cc:654] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:47.130744 7257 tablet_bootstrap.cc:492] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Bootstrap starting.
I20260811 06:55:47.132594 7257 tablet_bootstrap.cc:654] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:47.135843 7252 tablet_bootstrap.cc:492] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Bootstrap starting.
I20260811 06:55:47.137800 7252 tablet_bootstrap.cc:654] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:47.139293 7253 tablet_bootstrap.cc:492] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: No bootstrap required, opened a new log
I20260811 06:55:47.139464 7253 ts_tablet_manager.cc:1397] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 06:55:47.139834 6305 catalog_manager.cc:4841] TS 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173): tablet 5771e939958248d79578fb33d2055692 (table test-table [id=53d47b8556274035b511240deeb0642c]) successfully deleted
I20260811 06:55:47.140432 7253 raft_consensus.cc:348] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.140842 7253 raft_consensus.cc:374] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:47.142771 7285 ts_tablet_manager.cc:1905] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:47.141016 7253 raft_consensus.cc:729] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d5e9ec5e6e411d853a7ea47e56e6dd, State: Initialized, Role: FOLLOWER
I20260811 06:55:47.143461 7253 consensus_queue.cc:260] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.144979 7253 ts_tablet_manager.cc:1428] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:55:47.156389 7257 tablet_bootstrap.cc:492] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: No bootstrap required, opened a new log
I20260811 06:55:47.156579 7257 ts_tablet_manager.cc:1397] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.012s
I20260811 06:55:47.156701 7252 tablet_bootstrap.cc:492] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: No bootstrap required, opened a new log
I20260811 06:55:47.156899 7252 ts_tablet_manager.cc:1397] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Time spent bootstrapping tablet: real 0.021s user 0.007s sys 0.003s
I20260811 06:55:47.157595 7257 raft_consensus.cc:348] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.157862 7257 raft_consensus.cc:374] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:47.157981 7257 raft_consensus.cc:729] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088141433f5043e996d0b41085ebb007, State: Initialized, Role: FOLLOWER
I20260811 06:55:47.157743 7252 raft_consensus.cc:348] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.158201 7252 raft_consensus.cc:374] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:47.158358 7252 raft_consensus.cc:729] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c4705025444296945a6a9c015dc5c3, State: Initialized, Role: FOLLOWER
I20260811 06:55:47.158346 7257 consensus_queue.cc:260] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.158895 7284 ts_tablet_manager.cc:1918] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:55:47.158975 7257 ts_tablet_manager.cc:1428] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:55:47.159088 7284 log.cc:1196] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/5771e939958248d79578fb33d2055692
I20260811 06:55:47.158857 7252 consensus_queue.cc:260] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.159792 7284 ts_tablet_manager.cc:1939] T 5771e939958248d79578fb33d2055692 P 088141433f5043e996d0b41085ebb007: Deleting consensus metadata
I20260811 06:55:47.160840 7252 ts_tablet_manager.cc:1428] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260811 06:55:47.161441 7285 ts_tablet_manager.cc:1918] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:55:47.161595 7285 log.cc:1196] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/5771e939958248d79578fb33d2055692
I20260811 06:55:47.161772 6313 catalog_manager.cc:4841] TS 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001): tablet 5771e939958248d79578fb33d2055692 (table test-table [id=53d47b8556274035b511240deeb0642c]) successfully deleted
I20260811 06:55:47.162168 7285 ts_tablet_manager.cc:1939] T 5771e939958248d79578fb33d2055692 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Deleting consensus metadata
I20260811 06:55:47.163946 6313 catalog_manager.cc:4841] TS 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3:39927): tablet 5771e939958248d79578fb33d2055692 (table test-table [id=53d47b8556274035b511240deeb0642c]) successfully deleted
I20260811 06:55:47.210917 7261 raft_consensus.cc:482] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:47.211172 7261 raft_consensus.cc:504] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.212749 7261 leader_election.cc:288] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001), 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173)
I20260811 06:55:47.219734 7244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.213225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:43073 (local address 127.4.40.2:43173)
0811 06:55:47.213418 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.213426 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.213724 (+ 298us) server_negotiation.cc:407] Connection header received
0811 06:55:47.214013 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.214019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.214099 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.214169 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:47.214864 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.216046 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.217999 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.218777 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.218917 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.219030 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.219303 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.219430 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 06:55:47.219810 7296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.213034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:43173 (local address 127.4.40.3:43073)
0811 06:55:47.213609 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:47.213630 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:47.213797 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:47.214433 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:47.214437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:47.214459 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:47.214692 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.214703 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.216266 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.216271 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.217831 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.217844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.218951 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.218957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.219142 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.219167 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.219276 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.219403 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":1,"thread_start_us":375,"threads_started":1}
I20260811 06:55:47.220713 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90a38c5c1b14a61be23108fd4ce10fb" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c4705025444296945a6a9c015dc5c3" is_pre_election: true
I20260811 06:55:47.220983 6872 raft_consensus.cc:2454] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 0.
I20260811 06:55:47.221542 6991 leader_election.cc:302] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d5e9ec5e6e411d853a7ea47e56e6dd, 87c4705025444296945a6a9c015dc5c3; no voters:
I20260811 06:55:47.222043 7261 raft_consensus.cc:2781] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:47.222131 7250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.213289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38177 (local address 127.4.40.1:43001)
0811 06:55:47.213496 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.213503 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.214272 (+ 769us) server_negotiation.cc:407] Connection header received
0811 06:55:47.214481 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.214486 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.214568 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.214639 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:47.215429 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.216572 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.219484 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.220945 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.221113 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.221223 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.221650 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.221789 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 06:55:47.222196 7261 raft_consensus.cc:482] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:47.222303 7261 raft_consensus.cc:3037] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.222507 7297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.213040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:43001 (local address 127.4.40.3:38177)
0811 06:55:47.214147 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:47.214166 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:47.214275 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:47.214831 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:47.214836 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:47.214860 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:47.215203 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.215215 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.216715 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.216722 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.219268 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.219286 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.221265 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.221275 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.221473 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.221497 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.221614 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.221816 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":534,"thread_start_us":271,"threads_started":1}
I20260811 06:55:47.223381 6620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90a38c5c1b14a61be23108fd4ce10fb" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088141433f5043e996d0b41085ebb007" is_pre_election: true
I20260811 06:55:47.223829 6620 raft_consensus.cc:2454] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 0.
I20260811 06:55:47.225941 7261 raft_consensus.cc:504] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.227317 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90a38c5c1b14a61be23108fd4ce10fb" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c4705025444296945a6a9c015dc5c3"
I20260811 06:55:47.227573 6872 raft_consensus.cc:3037] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.227608 6620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f90a38c5c1b14a61be23108fd4ce10fb" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088141433f5043e996d0b41085ebb007"
I20260811 06:55:47.227877 6620 raft_consensus.cc:3037] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.228195 7261 leader_election.cc:288] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 election: Requested vote from peers 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001), 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173)
I20260811 06:55:47.230526 6620 raft_consensus.cc:2454] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 1.
I20260811 06:55:47.230944 6872 raft_consensus.cc:2454] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 1.
I20260811 06:55:47.230954 6997 leader_election.cc:302] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d5e9ec5e6e411d853a7ea47e56e6dd, 088141433f5043e996d0b41085ebb007; no voters:
I20260811 06:55:47.231279 7261 raft_consensus.cc:2781] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:47.231762 7261 raft_consensus.cc:686] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 LEADER]: Becoming Leader. State: Replica: 00d5e9ec5e6e411d853a7ea47e56e6dd, State: Running, Role: LEADER
I20260811 06:55:47.232228 7261 consensus_queue.cc:237] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.236290 6340 catalog_manager.cc:5495] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3). New cstate: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } health_report { overall_health: HEALTHY } } }
Aug 11 06:55:47 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:47.314725 7313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.298336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37461 (local address 127.0.0.1:34550)
0811 06:55:47.299540 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:47.299562 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:47.299737 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:47.300169 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:47.300173 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:47.300887 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:47.301140 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.301149 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.302789 (+ 1640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.302792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.303855 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.303867 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.304864 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.304878 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.305011 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.305710 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:47.305727 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:47.308404 (+ 2677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:47.312382 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:47.312392 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:47.312398 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:47.312794 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:47.313237 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:47.313242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:47.313245 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:47.313414 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:47.313997 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:47.314002 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:47.314142 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.314275 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.314464 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"thread_start_us":874,"threads_started":1}
I20260811 06:55:47.315143 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.298507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34550 (local address 127.4.40.62:37461)
0811 06:55:47.299396 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.299404 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.299698 (+ 294us) server_negotiation.cc:407] Connection header received
0811 06:55:47.299856 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.299863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.299950 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.300078 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:47.301332 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.302649 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.303986 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.304652 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.304802 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.308622 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:47.308629 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:47.308634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:47.308672 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:47.312239 (+ 3567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.312912 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.312917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.312921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.313100 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.313518 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.313523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.313527 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.313862 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:47.314012 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.314404 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.314644 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":465,"threads_started":1}
I20260811 06:55:47.320816 6340 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34550:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:55:47.322120 6340 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:47.329190 6589 tablet_service.cc:1467] Processing CreateTablet for tablet f3d4329dbd424a6ea7bd4b3466ba4dc4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e681b1b3992452db60e567d5f16cc7c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:55:47.329298 6840 tablet_service.cc:1467] Processing CreateTablet for tablet f3d4329dbd424a6ea7bd4b3466ba4dc4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e681b1b3992452db60e567d5f16cc7c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:55:47.329813 6840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d4329dbd424a6ea7bd4b3466ba4dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:47.329823 6589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d4329dbd424a6ea7bd4b3466ba4dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:47.337669 7257 tablet_bootstrap.cc:492] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: Bootstrap starting.
I20260811 06:55:47.339298 7252 tablet_bootstrap.cc:492] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: Bootstrap starting.
I20260811 06:55:47.339439 7257 tablet_bootstrap.cc:654] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:47.341084 7252 tablet_bootstrap.cc:654] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:47.341219 7072 tablet_service.cc:1467] Processing CreateTablet for tablet f3d4329dbd424a6ea7bd4b3466ba4dc4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e681b1b3992452db60e567d5f16cc7c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:55:47.341749 7072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3d4329dbd424a6ea7bd4b3466ba4dc4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:47.343441 7252 tablet_bootstrap.cc:492] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: No bootstrap required, opened a new log
I20260811 06:55:47.343614 7252 ts_tablet_manager.cc:1397] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 06:55:47.344048 7257 tablet_bootstrap.cc:492] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: No bootstrap required, opened a new log
I20260811 06:55:47.344238 7257 ts_tablet_manager.cc:1397] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260811 06:55:47.345057 7257 raft_consensus.cc:348] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.345326 7257 raft_consensus.cc:374] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:47.345425 7257 raft_consensus.cc:729] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 088141433f5043e996d0b41085ebb007, State: Initialized, Role: FOLLOWER
I20260811 06:55:47.345438 7252 raft_consensus.cc:348] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.345736 7252 raft_consensus.cc:374] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:47.345842 7252 raft_consensus.cc:729] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87c4705025444296945a6a9c015dc5c3, State: Initialized, Role: FOLLOWER
I20260811 06:55:47.345675 7257 consensus_queue.cc:260] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.346125 7252 consensus_queue.cc:260] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.346331 7262 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.346622 7262 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.346637 7259 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.346976 7259 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.347260 7257 ts_tablet_manager.cc:1428] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 06:55:47.348040 7252 ts_tablet_manager.cc:1428] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 06:55:47.352509 7253 tablet_bootstrap.cc:492] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Bootstrap starting.
I20260811 06:55:47.354197 7253 tablet_bootstrap.cc:654] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:47.356297 7253 tablet_bootstrap.cc:492] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: No bootstrap required, opened a new log
I20260811 06:55:47.356472 7253 ts_tablet_manager.cc:1397] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:55:47.357210 7253 raft_consensus.cc:348] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.357445 7253 raft_consensus.cc:374] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:47.357522 7253 raft_consensus.cc:729] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d5e9ec5e6e411d853a7ea47e56e6dd, State: Initialized, Role: FOLLOWER
I20260811 06:55:47.357730 7253 consensus_queue.cc:260] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.358264 7253 ts_tablet_manager.cc:1428] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:55:47.358168 7261 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.358460 7261 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.545416 7261 raft_consensus.cc:482] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:47.545647 7261 raft_consensus.cc:504] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.546331 7261 leader_election.cc:288] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001), 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173)
I20260811 06:55:47.546876 6620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d4329dbd424a6ea7bd4b3466ba4dc4" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088141433f5043e996d0b41085ebb007" is_pre_election: true
I20260811 06:55:47.547127 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d4329dbd424a6ea7bd4b3466ba4dc4" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c4705025444296945a6a9c015dc5c3" is_pre_election: true
I20260811 06:55:47.547207 6620 raft_consensus.cc:2454] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 0.
I20260811 06:55:47.547400 6872 raft_consensus.cc:2454] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 0.
I20260811 06:55:47.547662 6997 leader_election.cc:302] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d5e9ec5e6e411d853a7ea47e56e6dd, 088141433f5043e996d0b41085ebb007; no voters:
I20260811 06:55:47.548007 7261 raft_consensus.cc:2781] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:47.548178 7261 raft_consensus.cc:482] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:47.548323 7261 raft_consensus.cc:3037] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.550873 7261 raft_consensus.cc:504] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.551496 7261 leader_election.cc:288] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 election: Requested vote from peers 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001), 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173)
I20260811 06:55:47.551987 6872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d4329dbd424a6ea7bd4b3466ba4dc4" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87c4705025444296945a6a9c015dc5c3"
I20260811 06:55:47.552081 6620 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3d4329dbd424a6ea7bd4b3466ba4dc4" candidate_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "088141433f5043e996d0b41085ebb007"
I20260811 06:55:47.552266 6872 raft_consensus.cc:3037] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.552421 6620 raft_consensus.cc:3037] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:47.554366 6872 raft_consensus.cc:2454] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 1.
I20260811 06:55:47.554764 6991 leader_election.cc:302] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d5e9ec5e6e411d853a7ea47e56e6dd, 87c4705025444296945a6a9c015dc5c3; no voters:
I20260811 06:55:47.555092 7261 raft_consensus.cc:2781] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:47.555253 7261 raft_consensus.cc:686] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 LEADER]: Becoming Leader. State: Replica: 00d5e9ec5e6e411d853a7ea47e56e6dd, State: Running, Role: LEADER
I20260811 06:55:47.555519 7261 consensus_queue.cc:237] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } }
I20260811 06:55:47.556593 6620 raft_consensus.cc:2454] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d5e9ec5e6e411d853a7ea47e56e6dd in term 1.
I20260811 06:55:47.557425 7299 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: TxnStatusTablet state changed. Reason: New leader 00d5e9ec5e6e411d853a7ea47e56e6dd. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.557742 7299 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:55:47.560169 6340 catalog_manager.cc:5495] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3). New cstate: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } health_report { overall_health: HEALTHY } } }
I20260811 06:55:47.561062 7333 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:55:47.561492 7333 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:55:47.600540 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.590101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45992 (local address 127.4.40.3:39927)
0811 06:55:47.592799 (+ 2698us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.592807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.592832 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:55:47.592951 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.592958 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.593106 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.593312 (+ 206us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:47.593771 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.595430 (+ 1659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.597902 (+ 2472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.598447 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.598572 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.599250 (+ 678us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:47.599824 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:47.599939 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.600188 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.600256 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2542,"thread_start_us":2390,"threads_started":1}
I20260811 06:55:47.600584 7313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.589139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:39927 (local address 127.0.0.1:45992)
0811 06:55:47.589602 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:47.589623 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:47.589764 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:47.593357 (+ 3593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:47.593363 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:47.593377 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:47.593603 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.593615 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.595594 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.595601 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.597739 (+ 2138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.597755 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.598628 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.598634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.598844 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.598939 (+ 95us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:47.599952 (+ 1013us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:47.599978 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.600074 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.600275 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286}
I20260811 06:55:47.605631 6872 raft_consensus.cc:1261] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Refusing update from remote peer 00d5e9ec5e6e411d853a7ea47e56e6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:47.606032 6620 raft_consensus.cc:1261] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Refusing update from remote peer 00d5e9ec5e6e411d853a7ea47e56e6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:47.606340 7261 consensus_queue.cc:1035] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:47.606868 7299 consensus_queue.cc:1035] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:47.617749 7259 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: TxnStatusTablet state changed. Reason: New leader 00d5e9ec5e6e411d853a7ea47e56e6dd. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.618145 7259 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.620077 7341 mvcc.cc:205] Tried to move back new op lower bound from 7317222799783866368 to 7317222799589167104. Current Snapshot: MvccSnapshot[applied={T|T < 7317222799783866368}]
I20260811 06:55:47.621781 7299 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.622131 7299 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:55:47.629736 7259 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.629984 7259 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 87c4705025444296945a6a9c015dc5c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.629931 7262 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: TxnStatusTablet state changed. Reason: New leader 00d5e9ec5e6e411d853a7ea47e56e6dd. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.630208 7262 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.632390 7261 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.632689 7261 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 00d5e9ec5e6e411d853a7ea47e56e6dd: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:55:47.641117 7262 tablet_replica.cc:439] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 } } peers { permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 } } peers { permanent_uuid: "00d5e9ec5e6e411d853a7ea47e56e6dd" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 39927 } } }
I20260811 06:55:47.641366 7262 tablet_replica.cc:442] T f3d4329dbd424a6ea7bd4b3466ba4dc4 P 088141433f5043e996d0b41085ebb007: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:47.651022 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.641217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45994 (local address 127.4.40.3:39927)
0811 06:55:47.642039 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.642047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.644303 (+ 2256us) server_negotiation.cc:407] Connection header received
0811 06:55:47.644492 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.644498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.644586 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.644679 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:47.645384 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.646571 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.648699 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.649502 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.649623 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.650110 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:47.650258 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:47.650351 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.650628 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.650721 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":686}
I20260811 06:55:47.671409 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.663504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:55237 (local address 127.4.40.3:39927)
0811 06:55:47.663693 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.663699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.664800 (+ 1101us) server_negotiation.cc:407] Connection header received
0811 06:55:47.664959 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.664965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.665069 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.665175 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:47.665768 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.667036 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.669257 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.669929 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.670051 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.670596 (+ 545us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:47.670740 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:47.670828 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.670973 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.671056 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 06:55:47.671514 7360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.663541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:39927 (local address 127.4.40.3:55237)
0811 06:55:47.664675 (+ 1134us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:47.664693 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:47.664841 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:47.665190 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:47.665195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:47.665206 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:47.665581 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.665592 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.667177 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.667183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.669097 (+ 1914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:47.669112 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.670124 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.670130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:47.670336 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.670406 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:47.670843 (+ 437us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:47.670854 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:55:47.670927 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 06:55:47.671077 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":840,"threads_started":1}
I20260811 06:55:47.683995 6872 raft_consensus.cc:1261] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Refusing update from remote peer 00d5e9ec5e6e411d853a7ea47e56e6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:47.684281 6620 raft_consensus.cc:1261] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Refusing update from remote peer 00d5e9ec5e6e411d853a7ea47e56e6dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:47.684630 7299 consensus_queue.cc:1035] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "87c4705025444296945a6a9c015dc5c3" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:47.684892 7261 consensus_queue.cc:1035] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "088141433f5043e996d0b41085ebb007" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 43001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:47.696513 7343 mvcc.cc:205] Tried to move back new op lower bound from 7317222800108957696 to 7317222798266691584. Current Snapshot: MvccSnapshot[applied={T|T < 7317222800108957696}]
I20260811 06:55:47.759199 6340 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:47.759469 6340 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34546:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:47.762928 6340 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6d2df16f12d46d7867b85871614acbe: Sending DeleteTablet for 3 replicas of tablet f90a38c5c1b14a61be23108fd4ce10fb
I20260811 06:55:47.763672 6589 tablet_service.cc:1514] Processing DeleteTablet for tablet f90a38c5c1b14a61be23108fd4ce10fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:47 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:32838
I20260811 06:55:47.764130 6840 tablet_service.cc:1514] Processing DeleteTablet for tablet f90a38c5c1b14a61be23108fd4ce10fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:47 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:33266
I20260811 06:55:47.771001 7072 tablet_service.cc:1514] Processing DeleteTablet for tablet f90a38c5c1b14a61be23108fd4ce10fb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:47 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:45988
I20260811 06:55:47.775856 7371 tablet_replica.cc:330] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: stopping tablet replica
I20260811 06:55:47.776228 7371 raft_consensus.cc:2229] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:47.776635 7371 raft_consensus.cc:2258] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:47.778074 7371 ts_tablet_manager.cc:1905] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:47.785951 7375 tablet_replica.cc:330] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: stopping tablet replica
I20260811 06:55:47.786290 7375 raft_consensus.cc:2229] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:55:47.786759 7372 tablet_replica.cc:330] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: stopping tablet replica
I20260811 06:55:47.787406 7375 raft_consensus.cc:2258] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:47.787717 7372 raft_consensus.cc:2229] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:47.787781 7371 ts_tablet_manager.cc:1918] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:47.787992 7372 raft_consensus.cc:2258] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:47.788462 7371 log.cc:1196] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/f90a38c5c1b14a61be23108fd4ce10fb
I20260811 06:55:47.789201 7371 ts_tablet_manager.cc:1939] T f90a38c5c1b14a61be23108fd4ce10fb P 088141433f5043e996d0b41085ebb007: Deleting consensus metadata
I20260811 06:55:47.789253 7375 ts_tablet_manager.cc:1905] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:47.790304 7372 ts_tablet_manager.cc:1905] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:47.790561 6313 catalog_manager.cc:4841] TS 088141433f5043e996d0b41085ebb007 (127.4.40.1:43001): tablet f90a38c5c1b14a61be23108fd4ce10fb (table test-table [id=ff30364197aa40738a9aba068f8eeca6]) successfully deleted
Aug 11 06:55:47 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:47.800290 7372 ts_tablet_manager.cc:1918] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:47.800463 7372 log.cc:1196] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/f90a38c5c1b14a61be23108fd4ce10fb
I20260811 06:55:47.800997 7372 ts_tablet_manager.cc:1939] T f90a38c5c1b14a61be23108fd4ce10fb P 87c4705025444296945a6a9c015dc5c3: Deleting consensus metadata
I20260811 06:55:47.801225 7375 ts_tablet_manager.cc:1918] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:47.801433 7375 log.cc:1196] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/f90a38c5c1b14a61be23108fd4ce10fb
I20260811 06:55:47.802210 6305 catalog_manager.cc:4841] TS 87c4705025444296945a6a9c015dc5c3 (127.4.40.2:43173): tablet f90a38c5c1b14a61be23108fd4ce10fb (table test-table [id=ff30364197aa40738a9aba068f8eeca6]) successfully deleted
I20260811 06:55:47.802016 7375 ts_tablet_manager.cc:1939] T f90a38c5c1b14a61be23108fd4ce10fb P 00d5e9ec5e6e411d853a7ea47e56e6dd: Deleting consensus metadata
I20260811 06:55:47.803745 6313 catalog_manager.cc:4841] TS 00d5e9ec5e6e411d853a7ea47e56e6dd (127.4.40.3:39927): tablet f90a38c5c1b14a61be23108fd4ce10fb (table test-table [id=ff30364197aa40738a9aba068f8eeca6]) successfully deleted
I20260811 06:55:47.808003 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.778988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32850 (local address 127.4.40.1:43001)
0811 06:55:47.788340 (+ 9352us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.788351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.790208 (+ 1857us) server_negotiation.cc:407] Connection header received
0811 06:55:47.790872 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.790876 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.790943 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.791065 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:47.792724 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.793478 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.794910 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.795505 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.795649 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.801428 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:47.801443 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:47.801461 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:47.801498 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:47.805171 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.805838 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.805847 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.805851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.805996 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.806410 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.806415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.806419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.806771 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:47.807089 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.807394 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.807644 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9187,"thread_start_us":9050,"threads_started":1}
W20260811 06:55:47.808769 6702 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32850
I20260811 06:55:47.825248 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.811644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.4.40.62:37461)
0811 06:55:47.811834 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.811839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.812335 (+ 496us) server_negotiation.cc:407] Connection header received
0811 06:55:47.812436 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.812442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.812542 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.812646 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:47.813815 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.814924 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.816146 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.816857 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.816994 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.819142 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:47.819148 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:47.819153 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:47.819192 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:47.822620 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.823212 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.823217 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.823221 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.823383 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.823727 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.823733 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.823736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.824173 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:47.824432 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.824676 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.824923 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 06:55:47.825906 6340 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34564
I20260811 06:55:47.828238 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6477
I20260811 06:55:47.845343 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.830012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32864 (local address 127.4.40.1:43001)
0811 06:55:47.830281 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.830289 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.832618 (+ 2329us) server_negotiation.cc:407] Connection header received
0811 06:55:47.832775 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.832782 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.832859 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.832969 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:47.834021 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.835116 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.836328 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.836947 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.837068 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.839138 (+ 2070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:47.839145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:47.839150 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:47.839185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:47.842644 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.843303 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.843309 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.843313 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.843480 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.843879 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.843884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.843887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.844253 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:47.844535 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.844749 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.844985 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 06:55:47.845989 6702 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32864
W20260811 06:55:47.846553 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6477: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:55:47.846738 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6477
I20260811 06:55:47.892298 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6717
Aug 11 06:55:47 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:55:47.908349 7383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.893137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33272 (local address 127.4.40.2:43173)
0811 06:55:47.893993 (+ 856us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.893998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.894024 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:55:47.894081 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.894085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.894165 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.894328 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:47.895603 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.896425 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.897630 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.898072 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.898209 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.902548 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:47.902562 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:47.902576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:47.902607 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:47.905506 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.906201 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.906211 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.906221 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.906355 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.906743 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.906748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.906753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.907090 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:47.907429 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.907731 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.907961 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":581,"threads_started":1}
W20260811 06:55:47.909212 6951 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33272
W20260811 06:55:47.909859 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6717: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:55:47.910043 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6717
I20260811 06:55:47.944653 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6967
Aug 11 06:55:47 dist-test-slave-7vjv krb5kdc[6251](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431346, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:55:47.960743 7340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:47.945616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46008 (local address 127.4.40.3:39927)
0811 06:55:47.945867 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:47.945877 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:47.945897 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:55:47.945968 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:47.945972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:47.946046 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:47.946181 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:47.947396 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.948239 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.949592 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:47.950166 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:47.950316 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:47.954871 (+ 4555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:47.954883 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:47.954896 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:47.954927 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:47.957929 (+ 3002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.958602 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.958610 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.958612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.958756 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:47.959151 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:47.959157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:47.959162 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:47.959528 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:47.959904 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:47.960208 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 06:55:47.960432 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 06:55:47.961733 7211 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46008
W20260811 06:55:47.962519 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6967: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:55:47.962682 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6967
I20260811 06:55:47.992632 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6275
I20260811 06:55:47.993891 6416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:48.111656 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6275
2026-08-11T06:55:48Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5164 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:55:48 dist-test-slave-7vjv krb5kdc[7387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:55:48 dist-test-slave-7vjv krb5kdc[7387](info): set up 2 sockets
Aug 11 06:55:48 dist-test-slave-7vjv krb5kdc[7387](info): commencing operation
krb5kdc: starting...
W20260811 06:55:49.456450 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:55:49 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:55:49.672230 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:35193
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:35193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:49.999933 7415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:50.000324 7415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:50.000505 7415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:50.007746 7415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:55:50.007886 7415 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:55:50.007979 7415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:50.008086 7415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:50.008190 7415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:55:50.008283 7415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:55:50.018093 7415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:35193
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:35193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:50.019435 7415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:50.020866 7415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:50.032729 7426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:50.032747 7423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:50.032653 7422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:50.033862 7415 server_base.cc:1034] running on GCE node
I20260811 06:55:50.034610 7415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:50.035920 7415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:50.037140 7415 hybrid_clock.cc:648] HybridClock initialized: now 1786431350037066 us; error 43 us; skew 500 ppm
I20260811 06:55:50.037423 7415 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:50 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431350, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:50.040549 7415 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:55:50.042433 7415 webserver.cc:460] Webserver started at http://127.4.40.62:43733/ using document root <none> and password file <none>
I20260811 06:55:50.043372 7415 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:50.043552 7415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:50.043953 7415 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:50.048731 7415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "94bee6203b364a37b1a6c1e2652ed4a1"
format_stamp: "Formatted at 2026-08-11 06:55:50 on dist-test-slave-7vjv"
server_key: "4a41614ef1e1c7f3ed4db3764e1633b2"
server_key_iv: "c765f59144ee8ef85bc47691994eaf0d"
server_key_version: "encryptionkey@0"
I20260811 06:55:50.049669 7415 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "94bee6203b364a37b1a6c1e2652ed4a1"
format_stamp: "Formatted at 2026-08-11 06:55:50 on dist-test-slave-7vjv"
server_key: "4a41614ef1e1c7f3ed4db3764e1633b2"
server_key_iv: "c765f59144ee8ef85bc47691994eaf0d"
server_key_version: "encryptionkey@0"
I20260811 06:55:50.055721 7415 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 06:55:50.059139 7441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:50.060484 7415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:50.060668 7415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "94bee6203b364a37b1a6c1e2652ed4a1"
format_stamp: "Formatted at 2026-08-11 06:55:50 on dist-test-slave-7vjv"
server_key: "4a41614ef1e1c7f3ed4db3764e1633b2"
server_key_iv: "c765f59144ee8ef85bc47691994eaf0d"
server_key_version: "encryptionkey@0"
I20260811 06:55:50.060894 7415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:50.100497 7415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:50.105149 7415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:50.105430 7415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:50.127245 7415 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:35193
I20260811 06:55:50.127288 7560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:35193 every 8 connection(s)
I20260811 06:55:50.128679 7415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:55:50.132267 7562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:50.134244 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7415
I20260811 06:55:50.134451 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:55:50.135140 4256 external_mini_cluster.cc:1428] Setting key 606b4b64dbcbedd9c767995c643c1998
I20260811 06:55:50.145305 7562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Bootstrap starting.
I20260811 06:55:50.148855 7562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:50.149852 7562 log.cc:824] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:55:50 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:50.153115 7562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: No bootstrap required, opened a new log
I20260811 06:55:50.160161 7566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:50.137436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42830 (local address 127.4.40.62:35193)
0811 06:55:50.138281 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:50.138297 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:50.138350 (+ 53us) server_negotiation.cc:407] Connection header received
0811 06:55:50.140734 (+ 2384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:50.140762 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:50.141249 (+ 487us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:50.141692 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:50.142985 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.144923 (+ 1938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.146186 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.147253 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.147453 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:50.151662 (+ 4209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:50.151672 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:50.151688 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:50.151736 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:50.155796 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:50.156499 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:50.156507 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:50.156515 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:50.156722 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:50.157094 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:50.157099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:50.157102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:50.157747 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:50.158023 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:50.158344 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 06:55:50.158770 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":396,"threads_started":1}
I20260811 06:55:50.163424 7562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35193 } }
I20260811 06:55:50.163784 7562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:50.163874 7562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94bee6203b364a37b1a6c1e2652ed4a1, State: Initialized, Role: FOLLOWER
I20260811 06:55:50.164436 7562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35193 } }
I20260811 06:55:50.164664 7562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:55:50.164760 7562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:55:50.164939 7562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:50.167104 7562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35193 } }
I20260811 06:55:50.167654 7562 leader_election.cc:302] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 94bee6203b364a37b1a6c1e2652ed4a1; no voters:
I20260811 06:55:50.168336 7562 leader_election.cc:288] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:55:50.168457 7568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:50.169209 7568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [term 1 LEADER]: Becoming Leader. State: Replica: 94bee6203b364a37b1a6c1e2652ed4a1, State: Running, Role: LEADER
I20260811 06:55:50.169744 7568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35193 } }
I20260811 06:55:50.170329 7562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:55:50.172689 7570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 94bee6203b364a37b1a6c1e2652ed4a1. Latest consensus state: current_term: 1 leader_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35193 } } }
I20260811 06:55:50.173075 7570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:50.172788 7569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "94bee6203b364a37b1a6c1e2652ed4a1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35193 } } }
I20260811 06:55:50.174197 7569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:55:50.175155 7579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:55:50.180336 7579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:55:50.189428 7579 catalog_manager.cc:1347] Generated new cluster ID: 5f840721d5f44d8cb90bcc87f59c1574
I20260811 06:55:50.189564 7579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:55:50.220532 7579 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:55:50.221834 7579 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:55:50.233417 7579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Generated new TSK 0
I20260811 06:55:50.234228 7579 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:55:50.297950 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:35193
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:50.666594 7608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:50.666858 7608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:50.666967 7608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:50.674746 7608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:50.674875 7608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:50.675063 7608 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:55:50.675168 7608 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:55:50.675284 7608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:55:50.685194 7608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:35193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:50.686458 7608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:50.687806 7608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:50.701467 7620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:50.701594 7617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:50.701571 7616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:50.702471 7608 server_base.cc:1034] running on GCE node
I20260811 06:55:50.703028 7608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:50.703861 7608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:50.705068 7608 hybrid_clock.cc:648] HybridClock initialized: now 1786431350705030 us; error 38 us; skew 500 ppm
I20260811 06:55:50.705379 7608 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:50 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431350, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:50.711951 7608 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:55:50.713527 7608 webserver.cc:460] Webserver started at http://127.4.40.1:35253/ using document root <none> and password file <none>
I20260811 06:55:50.714233 7608 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:50.714375 7608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:50.714705 7608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:50.717741 7608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
format_stamp: "Formatted at 2026-08-11 06:55:50 on dist-test-slave-7vjv"
server_key: "1fa5b5ff3ab1e5ed08bab9faa8578c20"
server_key_iv: "0b9bd75fc3c09c69186b66047ed7b21c"
server_key_version: "encryptionkey@0"
I20260811 06:55:50.718525 7608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
format_stamp: "Formatted at 2026-08-11 06:55:50 on dist-test-slave-7vjv"
server_key: "1fa5b5ff3ab1e5ed08bab9faa8578c20"
server_key_iv: "0b9bd75fc3c09c69186b66047ed7b21c"
server_key_version: "encryptionkey@0"
I20260811 06:55:50.723806 7608 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:55:50.727281 7633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:50.728673 7608 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:50.728859 7608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
format_stamp: "Formatted at 2026-08-11 06:55:50 on dist-test-slave-7vjv"
server_key: "1fa5b5ff3ab1e5ed08bab9faa8578c20"
server_key_iv: "0b9bd75fc3c09c69186b66047ed7b21c"
server_key_version: "encryptionkey@0"
I20260811 06:55:50.729139 7608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:50.741366 7608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:50.745693 7608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:50.745998 7608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:50.748701 7608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:50.748836 7608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:50.748996 7608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:50.749112 7608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:55:50 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431350, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:50.773728 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:50.749156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:50967 (local address 127.4.40.62:35193)
0811 06:55:50.750002 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:50.750009 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:50.750904 (+ 895us) server_negotiation.cc:407] Connection header received
0811 06:55:50.753046 (+ 2142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:50.753052 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:50.753148 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:50.753300 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:50.755456 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.756269 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.758911 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.759560 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.759735 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:50.765707 (+ 5972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:50.765717 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:50.765723 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:50.765769 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:50.769110 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:50.770297 (+ 1187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:50.770305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:50.770311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:50.770492 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:50.771219 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:50.771227 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:50.771232 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:50.771632 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:50.771804 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:50.772994 (+ 1190us) server_negotiation.cc:299] Negotiation successful
0811 06:55:50.773275 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":606,"threads_started":1}
I20260811 06:55:50.775614 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:50.749793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.4.40.1:50967)
0811 06:55:50.750720 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:50.750772 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:50.752758 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:50.753610 (+ 852us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:50.753620 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:50.754325 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:50.755268 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:50.755291 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.756444 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.756467 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:50.758646 (+ 2179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:50.758668 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.759898 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.759912 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:50.760296 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:50.761112 (+ 816us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:50.761156 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:50.765411 (+ 4255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:50.769273 (+ 3862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:50.769285 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:50.769309 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:50.770149 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:50.770636 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:50.770644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:50.770650 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:50.771050 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:50.771802 (+ 752us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:50.771814 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:50.772448 (+ 634us) client_negotiation.cc:769] Sending connection context
0811 06:55:50.772853 (+ 405us) client_negotiation.cc:240] Negotiation successful
0811 06:55:50.773537 (+ 684us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":416,"threads_started":1}
I20260811 06:55:50.793991 7608 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:40921
I20260811 06:55:50.794021 7853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:40921 every 8 connection(s)
I20260811 06:55:50.795591 7608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:55:50.798013 7855 heartbeater.cc:347] Connected to a master server at 127.4.40.62:35193
I20260811 06:55:50.798259 7855 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:50.799149 7855 heartbeater.cc:511] Master 127.4.40.62:35193 requested a full tablet report, sending...
I20260811 06:55:50.799527 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:50.783868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57807 (local address 127.4.40.62:35193)
0811 06:55:50.784069 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:50.784078 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:50.784896 (+ 818us) server_negotiation.cc:407] Connection header received
0811 06:55:50.785111 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:50.785120 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:50.785223 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:50.785366 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:50.786331 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.787578 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.789572 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.790167 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.790315 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:50.792607 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:50.792613 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:50.792618 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:50.792660 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:50.796187 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:50.797177 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:50.797182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:50.797203 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:50.797403 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:50.797852 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:50.797856 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:50.797860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:50.798267 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:50.798426 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:50.799025 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 06:55:50.799188 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:55:50.799767 7812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:50.783889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.4.40.1:57807)
0811 06:55:50.784757 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:50.784783 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:50.784965 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:50.785421 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:50.785426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:50.785875 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:50.786179 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:50.786189 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.787738 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.787745 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:50.789413 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:50.789431 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:50.790365 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:50.790371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:50.790530 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:50.791297 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:50.791324 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:50.792400 (+ 1076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:50.796368 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:50.796374 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:50.796381 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:50.797000 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:50.797558 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:50.797561 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:50.797564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:50.797745 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:50.798492 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:50.798497 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:50.798715 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 06:55:50.798883 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 06:55:50.799275 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"thread_start_us":535,"threads_started":1}
I20260811 06:55:50.800549 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7608
I20260811 06:55:50.800776 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:55:50.801540 4256 external_mini_cluster.cc:1428] Setting key 358f9fd5109bcfc7229093d0827da60a
I20260811 06:55:50.801682 7465 ts_manager.cc:194] Registered new tserver with Master: ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921)
I20260811 06:55:50.803588 7465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:50967
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:55:50.850728 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:35193
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:51.187196 7862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:51.187474 7862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:51.187651 7862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:51.194564 7862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:51.194681 7862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:51.194833 7862 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:55:51.194916 7862 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:55:51.195050 7862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:55:51.204463 7862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:35193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:51.205696 7862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:51.206974 7862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:51.220759 7873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:51.220752 7871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:51.221324 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:51.221572 7862 server_base.cc:1034] running on GCE node
I20260811 06:55:51.222177 7862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:51.223073 7862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:51.224257 7862 hybrid_clock.cc:648] HybridClock initialized: now 1786431351224220 us; error 33 us; skew 500 ppm
I20260811 06:55:51.224583 7862 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:51 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:51.227828 7862 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:55:51.229656 7862 webserver.cc:460] Webserver started at http://127.4.40.2:42607/ using document root <none> and password file <none>
I20260811 06:55:51.230474 7862 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:51.230615 7862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:51.230940 7862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:51.234133 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "5e06c69d9315418990ed5669643694f7"
format_stamp: "Formatted at 2026-08-11 06:55:51 on dist-test-slave-7vjv"
server_key: "f7ab801c9b9fa3bbb8a8b6ad7019c1d7"
server_key_iv: "3531996ef4e9d06ae257ca9c9578cf54"
server_key_version: "encryptionkey@0"
I20260811 06:55:51.234879 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "5e06c69d9315418990ed5669643694f7"
format_stamp: "Formatted at 2026-08-11 06:55:51 on dist-test-slave-7vjv"
server_key: "f7ab801c9b9fa3bbb8a8b6ad7019c1d7"
server_key_iv: "3531996ef4e9d06ae257ca9c9578cf54"
server_key_version: "encryptionkey@0"
I20260811 06:55:51.239898 7862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:55:51.243273 7889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:51.244653 7862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:51.244813 7862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "5e06c69d9315418990ed5669643694f7"
format_stamp: "Formatted at 2026-08-11 06:55:51 on dist-test-slave-7vjv"
server_key: "f7ab801c9b9fa3bbb8a8b6ad7019c1d7"
server_key_iv: "3531996ef4e9d06ae257ca9c9578cf54"
server_key_version: "encryptionkey@0"
I20260811 06:55:51.244977 7862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:51.258311 7862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:51.262156 7862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:51.262393 7862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:51.265106 7862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:51.265236 7862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:51.265365 7862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:51.265475 7862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:55:51 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:51.288458 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.265632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:51557 (local address 127.4.40.62:35193)
0811 06:55:51.265845 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.265851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.267320 (+ 1469us) server_negotiation.cc:407] Connection header received
0811 06:55:51.269731 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.269740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.269852 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.269963 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:51.272291 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.273353 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.274998 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.275600 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.275752 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.280631 (+ 4879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:51.280654 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:51.280661 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:51.280712 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:51.284528 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.285821 (+ 1293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.285828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.285834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.286021 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.286680 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.286687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.286692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.287000 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:51.287164 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.287856 (+ 692us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.288087 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:55:51.289988 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.266172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.4.40.2:51557)
0811 06:55:51.267133 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.267181 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.269448 (+ 2267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.270299 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.270314 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.271167 (+ 853us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:51.272064 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.272083 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.273528 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.273533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.274820 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.274832 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.275874 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.275889 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.276325 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.277322 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:51.277363 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:51.280349 (+ 2986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:51.284728 (+ 4379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.284742 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.284762 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.285610 (+ 848us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.286182 (+ 572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.286190 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.286197 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.286512 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.287146 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:51.287154 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:51.287470 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.287737 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.288285 (+ 548us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":477,"threads_started":1}
I20260811 06:55:51.312692 7862 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:33759
I20260811 06:55:51.312780 8105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:33759 every 8 connection(s)
I20260811 06:55:51.314369 7862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:55:51.314512 8038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.297513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.4.40.2:43763)
0811 06:55:51.298244 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.298272 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.298483 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.298917 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.298927 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.299716 (+ 789us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:51.300592 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.300614 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.302172 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.302176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.303694 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.303711 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.304888 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.304899 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.305160 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.305830 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:51.305860 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:51.307039 (+ 1179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:51.311375 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.311383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.311390 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.312021 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.312540 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.312548 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.312552 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.312744 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.313507 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:51.313512 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:51.313676 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.313796 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.314060 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":345,"threads_started":1}
I20260811 06:55:51.314569 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.297466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:43763 (local address 127.4.40.62:35193)
0811 06:55:51.297636 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.297643 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.298373 (+ 730us) server_negotiation.cc:407] Connection header received
0811 06:55:51.298623 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.298632 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.298749 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.298859 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:51.300760 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.302028 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.303898 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.304644 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.304787 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.307254 (+ 2467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:51.307264 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:51.307272 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:51.307320 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:51.311168 (+ 3848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.312224 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.312230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.312233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.312388 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.312911 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.312917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.312920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.313369 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:51.313526 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.313952 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.314209 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 06:55:51.317961 8106 heartbeater.cc:347] Connected to a master server at 127.4.40.62:35193
I20260811 06:55:51.318222 8106 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:51.319093 8106 heartbeater.cc:511] Master 127.4.40.62:35193 requested a full tablet report, sending...
I20260811 06:55:51.320526 7465 ts_manager.cc:194] Registered new tserver with Master: 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759)
I20260811 06:55:51.320545 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7862
I20260811 06:55:51.320713 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:55:51.321311 4256 external_mini_cluster.cc:1428] Setting key dd81aa36b1b5899192829c875a33ebfd
I20260811 06:55:51.321607 7465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:51557
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:55:51.370945 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:35193
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42015
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:51.713907 8112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:51.714178 8112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:51.714282 8112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:51.721542 8112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:51.721648 8112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:51.721735 8112 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 06:55:51.721786 8112 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:55:51.721865 8112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:55:51.731957 8112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42015
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:35193
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:51.733419 8112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:51.734855 8112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:51.748641 8120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:51.748641 8121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:51.748669 8123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:51.749976 8112 server_base.cc:1034] running on GCE node
I20260811 06:55:51.750628 8112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:51.751624 8112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:51.752882 8112 hybrid_clock.cc:648] HybridClock initialized: now 1786431351752818 us; error 46 us; skew 500 ppm
I20260811 06:55:51.753219 8112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:51 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:51.756597 8112 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:55:51.758137 8112 webserver.cc:460] Webserver started at http://127.4.40.3:42081/ using document root <none> and password file <none>
I20260811 06:55:51.758872 8112 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:51.759037 8112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:51.759383 8112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:51.762621 8112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "89f61caf65d5416abeca163a013a4097"
format_stamp: "Formatted at 2026-08-11 06:55:51 on dist-test-slave-7vjv"
server_key: "cf4dad80871c64010d7eabcf27a84131"
server_key_iv: "11088c9e01b2e05bc4d4e6e447e7fad5"
server_key_version: "encryptionkey@0"
I20260811 06:55:51.763506 8112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "89f61caf65d5416abeca163a013a4097"
format_stamp: "Formatted at 2026-08-11 06:55:51 on dist-test-slave-7vjv"
server_key: "cf4dad80871c64010d7eabcf27a84131"
server_key_iv: "11088c9e01b2e05bc4d4e6e447e7fad5"
server_key_version: "encryptionkey@0"
I20260811 06:55:51.769114 8112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 06:55:51.772758 8135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:51.774036 8112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:55:51.774219 8112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "89f61caf65d5416abeca163a013a4097"
format_stamp: "Formatted at 2026-08-11 06:55:51 on dist-test-slave-7vjv"
server_key: "cf4dad80871c64010d7eabcf27a84131"
server_key_iv: "11088c9e01b2e05bc4d4e6e447e7fad5"
server_key_version: "encryptionkey@0"
I20260811 06:55:51.774447 8112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:51.803193 8112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:51.806205 7855 heartbeater.cc:503] Master 127.4.40.62:35193 was elected leader, sending a full tablet report...
I20260811 06:55:51.807703 8112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:51.808094 8112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:51.811136 8112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:55:51.811261 8112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:51.811389 8112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:55:51.811487 8112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:55:51 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431351, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:51.836875 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.811301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:33791 (local address 127.4.40.62:35193)
0811 06:55:51.811539 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.811546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.813179 (+ 1633us) server_negotiation.cc:407] Connection header received
0811 06:55:51.816284 (+ 3105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.816288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.816357 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.816441 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:51.819029 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.820169 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.823591 (+ 3422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.823991 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.824159 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.830000 (+ 5841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:51.830021 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:51.830028 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:51.830073 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:51.833071 (+ 2998us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.834245 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.834249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.834252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.834383 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.834921 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.834924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.834926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.835193 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:51.835305 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.836381 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.836617 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:55:51.838937 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.811919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.4.40.3:33791)
0811 06:55:51.812987 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.813046 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.815930 (+ 2884us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.816758 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.816768 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.817703 (+ 935us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:51.818766 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.818795 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.820413 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.820426 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.823410 (+ 2984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.823431 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.824296 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.824306 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.824673 (+ 367us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.825652 (+ 979us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:51.825686 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:51.829732 (+ 4046us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:51.833267 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.833280 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.833300 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.834115 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.834502 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.834510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.834516 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.834804 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.835326 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:51.835346 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:51.835861 (+ 515us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.836265 (+ 404us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.836946 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":377,"threads_started":1}
I20260811 06:55:51.863165 8112 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:44239
I20260811 06:55:51.863261 8356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:44239 every 8 connection(s)
I20260811 06:55:51.864910 8112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:55:51.867216 8359 heartbeater.cc:347] Connected to a master server at 127.4.40.62:35193
I20260811 06:55:51.867847 8359 heartbeater.cc:464] Registering TS with master...
I20260811 06:55:51.868569 8359 heartbeater.cc:511] Master 127.4.40.62:35193 requested a full tablet report, sending...
I20260811 06:55:51.870024 7465 ts_manager.cc:194] Registered new tserver with Master: 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239)
I20260811 06:55:51.871214 7465 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:33791
I20260811 06:55:51.871917 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8112
I20260811 06:55:51.872144 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:55:51.872762 4256 external_mini_cluster.cc:1428] Setting key e56787aaad364e2b275481e50d826b1b
I20260811 06:55:51.873036 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.855777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:50211 (local address 127.4.40.62:35193)
0811 06:55:51.855995 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.856003 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.857213 (+ 1210us) server_negotiation.cc:407] Connection header received
0811 06:55:51.857545 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.857550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.857611 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.857691 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:51.859441 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.860834 (+ 1393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.863123 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.863676 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.863803 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.866531 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:51.866538 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:51.866543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:51.866580 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:51.869946 (+ 3366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.870804 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.870808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.870810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.870943 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.871349 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.871354 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.871356 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.871785 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:51.871930 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.872445 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.872675 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:55:51.873086 8320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.855345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.4.40.3:50211)
0811 06:55:51.857087 (+ 1742us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.857111 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.857375 (+ 264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.857765 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.857772 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.858656 (+ 884us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:51.859216 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.859236 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.861078 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.861084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.862944 (+ 1860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.862966 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.863947 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.863958 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.864157 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.864816 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:51.864843 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:51.866241 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:51.870115 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.870121 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.870126 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.870687 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.871054 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:51.871059 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:51.871062 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:51.871231 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:51.871891 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:51.871897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:51.872119 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.872297 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.872617 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1542,"thread_start_us":1339,"threads_started":1}
I20260811 06:55:51.876261 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:55:51 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:55:51 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:51.930281 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.917012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42834 (local address 127.4.40.62:35193)
0811 06:55:51.917276 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.917282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.917434 (+ 152us) server_negotiation.cc:407] Connection header received
0811 06:55:51.917600 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.917604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.917678 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.917763 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:51.919231 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.919944 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.921277 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.921678 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.921773 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.925498 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:51.925502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:51.925505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:51.925539 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:51.927920 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.928561 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.928565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.928567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.928687 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:51.929024 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:51.929027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:51.929029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:51.929268 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:51.929449 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.929709 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.929891 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 06:55:51.934329 7466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:51.941614 7466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:51.977532 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.953794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56042 (local address 127.4.40.2:33759)
0811 06:55:51.968693 (+ 14899us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.968705 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.968742 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:55:51.968861 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.968872 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.969191 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.970476 (+ 1285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:51.970709 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.971947 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.975101 (+ 3154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.976429 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.976894 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.977033 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.977088 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.977206 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14692,"thread_start_us":14525,"threads_started":1}
I20260811 06:55:51.977650 8380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.953492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:33759 (local address 127.0.0.1:56042)
0811 06:55:51.959591 (+ 6099us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.959651 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.959858 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.969488 (+ 9630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.969498 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.969550 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:51.970119 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.970135 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.972080 (+ 1945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.972087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.974590 (+ 2503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.974605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.976665 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.976671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.976869 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.976924 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.977055 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.977239 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5782,"mutex_wait_us":48,"thread_start_us":383,"threads_started":1}
I20260811 06:55:51.978718 8383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.954237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44239 (local address 127.0.0.1:46650)
0811 06:55:51.966869 (+ 12632us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.966885 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.967001 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.971584 (+ 4583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.971589 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.971613 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:51.971991 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.972003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.974080 (+ 2077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.974085 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.976047 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.976060 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.977947 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.977955 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.978152 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.978187 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.978263 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.978388 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12501,"thread_start_us":12358,"threads_started":1}
I20260811 06:55:51.979547 8385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.954856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:40921 (local address 127.0.0.1:34400)
0811 06:55:51.969472 (+ 14616us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:51.969486 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:51.969650 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:51.970353 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:51.970359 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:51.970386 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:51.971157 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.971174 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.972813 (+ 1639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.972819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.974544 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:51.974557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.976600 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.976609 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:51.976817 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.978942 (+ 2125us) client_negotiation.cc:769] Sending connection context
0811 06:55:51.979049 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:55:51.979183 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14506,"mutex_wait_us":6,"thread_start_us":14356,"threads_started":1}
I20260811 06:55:51.979635 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.955487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34400 (local address 127.4.40.1:40921)
0811 06:55:51.969029 (+ 13542us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.969041 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.969595 (+ 554us) server_negotiation.cc:407] Connection header received
0811 06:55:51.969825 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.969832 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.970055 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.970264 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:51.971425 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.972674 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.974720 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.976383 (+ 1663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.976553 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.976787 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.979153 (+ 2366us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.979316 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13343,"thread_start_us":13181,"threads_started":1}
I20260811 06:55:51.980014 8384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:51.954525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46650 (local address 127.4.40.3:44239)
0811 06:55:51.970547 (+ 16022us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:51.970555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:51.970633 (+ 78us) server_negotiation.cc:407] Connection header received
0811 06:55:51.970734 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:51.970740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:51.970941 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:51.971210 (+ 269us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:51.972217 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.973895 (+ 1678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.976219 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:51.977635 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:51.977763 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:51.977979 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:51.978396 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 06:55:51.978588 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15839,"thread_start_us":15611,"threads_started":1}
I20260811 06:55:51.981694 7975 tablet_service.cc:1467] Processing CreateTablet for tablet b23a88281c4f4cd3ace2d3b1845cc7b3 (DEFAULT_TABLE table=test-table [id=bfce81f57ab14468a1c7d717ea7a5a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:51.982887 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b23a88281c4f4cd3ace2d3b1845cc7b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:51.983274 8236 tablet_service.cc:1467] Processing CreateTablet for tablet b23a88281c4f4cd3ace2d3b1845cc7b3 (DEFAULT_TABLE table=test-table [id=bfce81f57ab14468a1c7d717ea7a5a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:51.983275 7722 tablet_service.cc:1467] Processing CreateTablet for tablet b23a88281c4f4cd3ace2d3b1845cc7b3 (DEFAULT_TABLE table=test-table [id=bfce81f57ab14468a1c7d717ea7a5a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:51.984643 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b23a88281c4f4cd3ace2d3b1845cc7b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:51.984658 8236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b23a88281c4f4cd3ace2d3b1845cc7b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:51.994117 8388 tablet_bootstrap.cc:492] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Bootstrap starting.
I20260811 06:55:51.997205 8388 tablet_bootstrap.cc:654] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:51.998342 8388 log.cc:824] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:52.004649 8389 tablet_bootstrap.cc:492] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Bootstrap starting.
I20260811 06:55:52.007833 8391 tablet_bootstrap.cc:492] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Bootstrap starting.
I20260811 06:55:52.009423 8389 tablet_bootstrap.cc:654] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.010846 8389 log.cc:824] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:52.011210 8388 tablet_bootstrap.cc:492] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: No bootstrap required, opened a new log
I20260811 06:55:52.011524 8391 tablet_bootstrap.cc:654] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.011540 8388 ts_tablet_manager.cc:1397] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.007s
I20260811 06:55:52.012709 8391 log.cc:824] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Log is configured to *not* fsync() on all Append() calls
I20260811 06:55:52.014089 8389 tablet_bootstrap.cc:492] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: No bootstrap required, opened a new log
I20260811 06:55:52.014341 8389 ts_tablet_manager.cc:1397] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:55:52.015321 8391 tablet_bootstrap.cc:492] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: No bootstrap required, opened a new log
I20260811 06:55:52.015579 8391 ts_tablet_manager.cc:1397] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 06:55:52.022221 8388 raft_consensus.cc:348] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.022676 8388 raft_consensus.cc:374] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.022779 8388 raft_consensus.cc:729] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e06c69d9315418990ed5669643694f7, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.022800 8391 raft_consensus.cc:348] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.023244 8391 raft_consensus.cc:374] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.023375 8391 raft_consensus.cc:729] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.023417 8388 consensus_queue.cc:260] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.024004 8391 consensus_queue.cc:260] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.024657 8106 heartbeater.cc:503] Master 127.4.40.62:35193 was elected leader, sending a full tablet report...
I20260811 06:55:52.024770 8389 raft_consensus.cc:348] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.025171 8389 raft_consensus.cc:374] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.025225 8388 ts_tablet_manager.cc:1428] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 06:55:52.025269 8389 raft_consensus.cc:729] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1223f3fa0b4f2384de9e88c67fd8c5, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.026243 8359 heartbeater.cc:503] Master 127.4.40.62:35193 was elected leader, sending a full tablet report...
I20260811 06:55:52.026801 8391 ts_tablet_manager.cc:1428] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Time spent starting tablet: real 0.011s user 0.003s sys 0.009s
I20260811 06:55:52.025974 8389 consensus_queue.cc:260] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.028414 8389 ts_tablet_manager.cc:1428] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
W20260811 06:55:52.048221 7856 tablet.cc:2369] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:55:52.067386 8107 tablet.cc:2369] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:55:52.117218 8362 tablet.cc:2369] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:55:52.296072 8397 raft_consensus.cc:482] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:52.296455 8397 raft_consensus.cc:504] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.298363 8397 leader_election.cc:288] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759), ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921)
I20260811 06:55:52.304363 8153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.298822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:33759 (local address 127.4.40.3:39377)
0811 06:55:52.299058 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.299082 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.299217 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.299783 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.299787 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.299811 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:52.300019 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.300031 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.301207 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.301212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.302449 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.302461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.303573 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.303578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.303764 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.303789 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.303904 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.304037 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:55:52.304363 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.298961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:39377 (local address 127.4.40.2:33759)
0811 06:55:52.299171 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.299177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.299197 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:55:52.299396 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.299402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.299477 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.299566 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:52.300251 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.301048 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.302653 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.303400 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.303562 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.303666 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.303931 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.304053 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:55:52.305418 8400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.299266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:40921 (local address 127.4.40.3:46651)
0811 06:55:52.299933 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.299949 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.300074 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.300524 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.300529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.300555 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:52.300926 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.300936 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.301987 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.301994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.303413 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.303427 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.304808 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.304814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.304993 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.305018 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.305090 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.305171 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":377,"threads_started":1}
I20260811 06:55:52.305609 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.299444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:46651 (local address 127.4.40.1:40921)
0811 06:55:52.299693 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.299700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.300046 (+ 346us) server_negotiation.cc:407] Connection header received
0811 06:55:52.300307 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.300311 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.300392 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.300508 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:52.301112 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.301837 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.303590 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.304588 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.304732 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.304854 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.305154 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.305318 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 06:55:52.305594 8024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a88281c4f4cd3ace2d3b1845cc7b3" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7" is_pre_election: true
I20260811 06:55:52.306264 8024 raft_consensus.cc:2454] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 0.
I20260811 06:55:52.306617 7757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a88281c4f4cd3ace2d3b1845cc7b3" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" is_pre_election: true
I20260811 06:55:52.307056 8136 leader_election.cc:302] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, 89f61caf65d5416abeca163a013a4097; no voters:
I20260811 06:55:52.307170 7757 raft_consensus.cc:2454] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 0.
I20260811 06:55:52.307492 8397 raft_consensus.cc:2781] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:52.307673 8397 raft_consensus.cc:482] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:52.307814 8397 raft_consensus.cc:3037] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.310365 8397 raft_consensus.cc:504] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.310990 8397 leader_election.cc:288] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 election: Requested vote from peers 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759), ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921)
I20260811 06:55:52.311476 8024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a88281c4f4cd3ace2d3b1845cc7b3" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7"
I20260811 06:55:52.311523 7757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b23a88281c4f4cd3ace2d3b1845cc7b3" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
I20260811 06:55:52.311729 8024 raft_consensus.cc:3037] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.311749 7757 raft_consensus.cc:3037] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.315348 7757 raft_consensus.cc:2454] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 1.
I20260811 06:55:52.315376 8024 raft_consensus.cc:2454] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 1.
I20260811 06:55:52.315842 8136 leader_election.cc:302] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, 89f61caf65d5416abeca163a013a4097; no voters:
I20260811 06:55:52.316195 8397 raft_consensus.cc:2781] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:52.316691 8397 raft_consensus.cc:686] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 1 LEADER]: Becoming Leader. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Running, Role: LEADER
I20260811 06:55:52.317488 8397 consensus_queue.cc:237] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.322389 7465 catalog_manager.cc:5495] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f61caf65d5416abeca163a013a4097 (127.4.40.3). New cstate: current_term: 1 leader_uuid: "89f61caf65d5416abeca163a013a4097" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } health_report { overall_health: HEALTHY } } }
Aug 11 06:55:52 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431350, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:52.376283 7648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.362435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42846 (local address 127.4.40.62:35193)
0811 06:55:52.362620 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.362627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.363141 (+ 514us) server_negotiation.cc:407] Connection header received
0811 06:55:52.363337 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.363342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.363406 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.363522 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:52.364864 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.365671 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.366920 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.367359 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.367499 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.370943 (+ 3444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:52.370950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:52.370954 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:52.370990 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:52.373519 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:52.374178 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:52.374185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:52.374188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:52.374334 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:52.374719 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:52.374722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:52.374724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:52.375021 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:52.375185 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.375609 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.375926 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":59}
I20260811 06:55:52.376330 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.362483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35193 (local address 127.0.0.1:42846)
0811 06:55:52.363035 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.363050 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.363218 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.363541 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.363545 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.364422 (+ 877us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:55:52.364716 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.364733 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.365797 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.365802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.366783 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.366795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.367544 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.367549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.367719 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.368397 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:55:52.368414 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:55:52.370756 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:55:52.373645 (+ 2889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:52.373659 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:52.373667 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:52.374058 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:52.374443 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:55:52.374448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:55:52.374451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:55:52.374588 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:55:52.375161 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:55:52.375169 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:55:52.375338 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.375506 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.375806 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":296,"threads_started":1}
I20260811 06:55:52.381539 7465 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:42846:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:55:52.382570 7465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:52.389036 7975 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2028d81c5245f18205359c7c59afbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0568a4f7a48a42609293839153e0f57c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:55:52.389051 7722 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2028d81c5245f18205359c7c59afbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0568a4f7a48a42609293839153e0f57c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:55:52.389245 8236 tablet_service.cc:1467] Processing CreateTablet for tablet 3d2028d81c5245f18205359c7c59afbd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0568a4f7a48a42609293839153e0f57c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:55:52.389899 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2028d81c5245f18205359c7c59afbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:52.389993 8236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2028d81c5245f18205359c7c59afbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:52.389997 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d2028d81c5245f18205359c7c59afbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:52.395924 8388 tablet_bootstrap.cc:492] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: Bootstrap starting.
I20260811 06:55:52.397188 8389 tablet_bootstrap.cc:492] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: Bootstrap starting.
I20260811 06:55:52.397459 8388 tablet_bootstrap.cc:654] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.398213 8391 tablet_bootstrap.cc:492] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: Bootstrap starting.
I20260811 06:55:52.399034 8389 tablet_bootstrap.cc:654] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.399492 8391 tablet_bootstrap.cc:654] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.401814 8389 tablet_bootstrap.cc:492] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: No bootstrap required, opened a new log
I20260811 06:55:52.401911 8391 tablet_bootstrap.cc:492] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: No bootstrap required, opened a new log
I20260811 06:55:52.401979 8389 ts_tablet_manager.cc:1397] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 06:55:52.402056 8391 ts_tablet_manager.cc:1397] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:55:52.402655 8389 raft_consensus.cc:348] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.402694 8391 raft_consensus.cc:348] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.402838 8389 raft_consensus.cc:374] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.402894 8391 raft_consensus.cc:374] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.402940 8389 raft_consensus.cc:729] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1223f3fa0b4f2384de9e88c67fd8c5, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.403005 8391 raft_consensus.cc:729] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.403214 8389 consensus_queue.cc:260] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.403245 8391 consensus_queue.cc:260] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.403625 8397 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.403817 8391 ts_tablet_manager.cc:1428] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:55:52.403827 8389 ts_tablet_manager.cc:1428] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:55:52.403937 8397 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.403662 8399 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.404070 8399 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.404867 8388 tablet_bootstrap.cc:492] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: No bootstrap required, opened a new log
I20260811 06:55:52.405049 8388 ts_tablet_manager.cc:1397] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 06:55:52.405879 8388 raft_consensus.cc:348] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.406157 8388 raft_consensus.cc:374] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.406263 8388 raft_consensus.cc:729] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e06c69d9315418990ed5669643694f7, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.406519 8388 consensus_queue.cc:260] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.407188 8388 ts_tablet_manager.cc:1428] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:55:52.407059 8395 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.407493 8395 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.514107 8399 raft_consensus.cc:482] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:52.514405 8399 raft_consensus.cc:504] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.516319 8399 leader_election.cc:288] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759), 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239)
I20260811 06:55:52.523387 8381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.516877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:55109 (local address 127.4.40.2:33759)
0811 06:55:52.517133 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.517143 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.517765 (+ 622us) server_negotiation.cc:407] Connection header received
0811 06:55:52.518121 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.518126 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.518198 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.518298 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:52.518891 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.519660 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.521638 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.522370 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.522514 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.522651 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.522953 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.523088 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:55:52.523471 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.516722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:33759 (local address 127.4.40.1:55109)
0811 06:55:52.517642 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.517663 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.517921 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.518319 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.518324 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.518364 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:52.518722 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.518731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.519796 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.519804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.521463 (+ 1659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.521475 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.522620 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.522626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.522777 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.522806 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.522923 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.523056 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"mutex_wait_us":49,"thread_start_us":1207,"threads_started":1}
I20260811 06:55:52.523664 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.516723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:51879 (local address 127.4.40.3:44239)
0811 06:55:52.517892 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.517898 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.518359 (+ 461us) server_negotiation.cc:407] Connection header received
0811 06:55:52.518594 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.518601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.518665 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.518765 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:52.519306 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.520012 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.521814 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.522650 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.522803 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.522917 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.523207 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.523345 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":920,"threads_started":1}
I20260811 06:55:52.523790 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.516725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44239 (local address 127.4.40.1:51879)
0811 06:55:52.518242 (+ 1517us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.518257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.518415 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.518785 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.518789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.518812 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:52.519139 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.519148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.520218 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.520222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.521652 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.521665 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.522872 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.522878 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.523045 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.523069 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.523182 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.523305 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1400,"mutex_wait_us":67,"thread_start_us":546,"threads_started":1}
I20260811 06:55:52.524626 8024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2028d81c5245f18205359c7c59afbd" candidate_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7" is_pre_election: true
I20260811 06:55:52.524675 8272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2028d81c5245f18205359c7c59afbd" candidate_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f61caf65d5416abeca163a013a4097" is_pre_election: true
I20260811 06:55:52.524915 8024 raft_consensus.cc:2454] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef1223f3fa0b4f2384de9e88c67fd8c5 in term 0.
I20260811 06:55:52.525101 8272 raft_consensus.cc:2454] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef1223f3fa0b4f2384de9e88c67fd8c5 in term 0.
I20260811 06:55:52.525434 7634 leader_election.cc:302] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, ef1223f3fa0b4f2384de9e88c67fd8c5; no voters:
I20260811 06:55:52.525863 8399 raft_consensus.cc:2781] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:52.525971 8399 raft_consensus.cc:482] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:52.526104 8399 raft_consensus.cc:3037] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.528311 8399 raft_consensus.cc:504] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.528916 8399 leader_election.cc:288] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [CANDIDATE]: Term 1 election: Requested vote from peers 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759), 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239)
I20260811 06:55:52.529397 8024 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2028d81c5245f18205359c7c59afbd" candidate_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7"
I20260811 06:55:52.529449 8272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d2028d81c5245f18205359c7c59afbd" candidate_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f61caf65d5416abeca163a013a4097"
I20260811 06:55:52.529616 8024 raft_consensus.cc:3037] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.529692 8272 raft_consensus.cc:3037] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.531653 8024 raft_consensus.cc:2454] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef1223f3fa0b4f2384de9e88c67fd8c5 in term 1.
I20260811 06:55:52.531858 8272 raft_consensus.cc:2454] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef1223f3fa0b4f2384de9e88c67fd8c5 in term 1.
I20260811 06:55:52.532152 7634 leader_election.cc:302] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, ef1223f3fa0b4f2384de9e88c67fd8c5; no voters:
I20260811 06:55:52.532428 8399 raft_consensus.cc:2781] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:52.532876 8399 raft_consensus.cc:686] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 LEADER]: Becoming Leader. State: Replica: ef1223f3fa0b4f2384de9e88c67fd8c5, State: Running, Role: LEADER
I20260811 06:55:52.533484 8399 consensus_queue.cc:237] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.537272 8425 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: TxnStatusTablet state changed. Reason: New leader ef1223f3fa0b4f2384de9e88c67fd8c5. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.537683 8425 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:55:52.538422 8432 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:55:52.538704 8432 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:55:52.538990 7465 catalog_manager.cc:5495] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1). New cstate: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } health_report { overall_health: UNKNOWN } } }
I20260811 06:55:52.559159 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.553256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34402 (local address 127.4.40.1:40921)
0811 06:55:52.553450 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.553455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.553531 (+ 76us) server_negotiation.cc:407] Connection header received
0811 06:55:52.553813 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.553820 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.553908 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.554019 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:52.554420 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.555169 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.557047 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.557454 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.557553 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.558086 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.558557 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.558656 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.558824 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.558890 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 06:55:52.559266 8416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.553170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:40921 (local address 127.0.0.1:34402)
0811 06:55:52.553433 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.553455 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.553608 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.554074 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.554079 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.554092 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:52.554286 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.554296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.555296 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.555302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.556913 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.556931 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.557623 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.557629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.557835 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.557906 (+ 71us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.558671 (+ 765us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.558701 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.558795 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.558956 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:55:52.567057 8272 raft_consensus.cc:1261] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Refusing update from remote peer ef1223f3fa0b4f2384de9e88c67fd8c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:52.567078 8024 raft_consensus.cc:1261] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Refusing update from remote peer ef1223f3fa0b4f2384de9e88c67fd8c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:52.568060 8425 consensus_queue.cc:1035] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:52.568787 8425 consensus_queue.cc:1035] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:52.584290 8436 mvcc.cc:205] Tried to move back new op lower bound from 7317222820103974912 to 7317222819981209600. Current Snapshot: MvccSnapshot[applied={T|T < 7317222820103974912}]
I20260811 06:55:52.585422 8397 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: TxnStatusTablet state changed. Reason: New leader ef1223f3fa0b4f2384de9e88c67fd8c5. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.585742 8397 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.587859 8399 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.588191 8399 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:55:52.593499 8444 mvcc.cc:205] Tried to move back new op lower bound from 7317222820103974912 to 7317222819981209600. Current Snapshot: MvccSnapshot[applied={T|T < 7317222820103974912}]
I20260811 06:55:52.597782 8395 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: TxnStatusTablet state changed. Reason: New leader ef1223f3fa0b4f2384de9e88c67fd8c5. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.598198 8395 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.602365 8397 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.602675 8397 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P 89f61caf65d5416abeca163a013a4097: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.609124 8425 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.609385 8425 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P ef1223f3fa0b4f2384de9e88c67fd8c5: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:55:52.611440 8446 mvcc.cc:205] Tried to move back new op lower bound from 7317222820103974912 to 7317222819981209600. Current Snapshot: MvccSnapshot[applied={T|T < 7317222820103974912}]
I20260811 06:55:52.613361 8395 tablet_replica.cc:439] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } } }
I20260811 06:55:52.613641 8395 tablet_replica.cc:442] T 3d2028d81c5245f18205359c7c59afbd P 5e06c69d9315418990ed5669643694f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:55:52.619382 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.611808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46656 (local address 127.4.40.3:44239)
0811 06:55:52.612077 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.612083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.612342 (+ 259us) server_negotiation.cc:407] Connection header received
0811 06:55:52.612660 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.612667 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.612796 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.612955 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:52.613513 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.614655 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.616337 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.616929 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.617089 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.617754 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.618464 (+ 710us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.618593 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.619109 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.619191 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 06:55:52.636667 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.630114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:48545 (local address 127.4.40.1:40921)
0811 06:55:52.630275 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.630279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.630836 (+ 557us) server_negotiation.cc:407] Connection header received
0811 06:55:52.631063 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.631067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.631151 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.631271 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:52.631765 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.632777 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.634518 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.634992 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.635184 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.635775 (+ 591us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.635882 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.636014 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.636233 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.636321 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:55:52.636701 8472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.630112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:40921 (local address 127.4.40.3:48545)
0811 06:55:52.630724 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.630744 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.630912 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.631268 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.631273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.631285 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:52.631604 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.631613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.632904 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.632911 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.634359 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.634373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.635233 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.635242 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.635429 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.635480 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.636028 (+ 548us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.636048 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.636193 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.636341 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":357,"threads_started":1}
I20260811 06:55:52.647953 8024 raft_consensus.cc:1261] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Refusing update from remote peer 89f61caf65d5416abeca163a013a4097: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:52.648270 7757 raft_consensus.cc:1261] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Refusing update from remote peer 89f61caf65d5416abeca163a013a4097: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:52.648598 8401 consensus_queue.cc:1035] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:52.649000 8397 consensus_queue.cc:1035] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:52.687130 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.679677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:41565 (local address 127.4.40.3:44239)
0811 06:55:52.679913 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.679922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.680722 (+ 800us) server_negotiation.cc:407] Connection header received
0811 06:55:52.680905 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.680908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.680970 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.681041 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:52.681683 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.682808 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.685058 (+ 2250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.685568 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.685699 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.686224 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.686341 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.686453 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.686755 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.686835 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 06:55:52.687270 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.679760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44239 (local address 127.4.40.1:41565)
0811 06:55:52.680611 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.680633 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.680787 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.681088 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.681093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.681105 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:52.681490 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.681504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.682992 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.682997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.684890 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.684905 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.685762 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.685769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.685951 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.686029 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:52.686580 (+ 551us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:52.686602 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.686696 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.686840 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":462,"threads_started":1}
I20260811 06:55:52.728760 7465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:52.729071 7465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:52.733234 7465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Sending DeleteTablet for 3 replicas of tablet b23a88281c4f4cd3ace2d3b1845cc7b3
I20260811 06:55:52.734570 7975 tablet_service.cc:1514] Processing DeleteTablet for tablet b23a88281c4f4cd3ace2d3b1845cc7b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:52 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56042
I20260811 06:55:52.734797 7722 tablet_service.cc:1514] Processing DeleteTablet for tablet b23a88281c4f4cd3ace2d3b1845cc7b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:52 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34400
I20260811 06:55:52.736276 8236 tablet_service.cc:1514] Processing DeleteTablet for tablet b23a88281c4f4cd3ace2d3b1845cc7b3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:52 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:46650
I20260811 06:55:52.738039 7465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42834:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:52.739187 7465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:52.740666 8491 tablet_replica.cc:330] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: stopping tablet replica
I20260811 06:55:52.741201 8491 raft_consensus.cc:2229] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:55:52.741886 8491 raft_consensus.cc:2258] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:52.749922 7974 tablet_service.cc:1467] Processing CreateTablet for tablet 6fae005747f04f2cb5282135bbc60ec0 (DEFAULT_TABLE table=test-table [id=cbce064e321c4538ada3e8c9e5af62ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:52.750154 8491 ts_tablet_manager.cc:1905] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:52.750511 7974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fae005747f04f2cb5282135bbc60ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:52.753002 7721 tablet_service.cc:1467] Processing CreateTablet for tablet 6fae005747f04f2cb5282135bbc60ec0 (DEFAULT_TABLE table=test-table [id=cbce064e321c4538ada3e8c9e5af62ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:52.753618 8488 tablet_replica.cc:330] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: stopping tablet replica
I20260811 06:55:52.753633 7721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fae005747f04f2cb5282135bbc60ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:52.754055 8488 raft_consensus.cc:2229] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:52.754448 8488 raft_consensus.cc:2258] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:52.755352 8236 tablet_service.cc:1467] Processing CreateTablet for tablet 6fae005747f04f2cb5282135bbc60ec0 (DEFAULT_TABLE table=test-table [id=cbce064e321c4538ada3e8c9e5af62ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:52.755769 8236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6fae005747f04f2cb5282135bbc60ec0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:52.756295 8488 ts_tablet_manager.cc:1905] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:52.760236 8389 tablet_bootstrap.cc:492] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Bootstrap starting.
I20260811 06:55:52.761888 8389 tablet_bootstrap.cc:654] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.764075 8488 ts_tablet_manager.cc:1918] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:52.764295 8488 log.cc:1196] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/b23a88281c4f4cd3ace2d3b1845cc7b3
I20260811 06:55:52.764879 8488 ts_tablet_manager.cc:1939] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 5e06c69d9315418990ed5669643694f7: Deleting consensus metadata
I20260811 06:55:52.766852 7442 catalog_manager.cc:4841] TS 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759): tablet b23a88281c4f4cd3ace2d3b1845cc7b3 (table test-table [id=bfce81f57ab14468a1c7d717ea7a5a60]) successfully deleted
I20260811 06:55:52.770090 8391 tablet_bootstrap.cc:492] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Bootstrap starting.
I20260811 06:55:52.772009 8391 tablet_bootstrap.cc:654] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.772500 8388 tablet_bootstrap.cc:492] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Bootstrap starting.
I20260811 06:55:52.774204 8388 tablet_bootstrap.cc:654] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:52.779968 8388 tablet_bootstrap.cc:492] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: No bootstrap required, opened a new log
I20260811 06:55:52.780171 8388 ts_tablet_manager.cc:1397] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 06:55:52.780937 8388 raft_consensus.cc:348] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.781176 8388 raft_consensus.cc:374] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.781245 8388 raft_consensus.cc:729] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e06c69d9315418990ed5669643694f7, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.781529 8388 consensus_queue.cc:260] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.784462 8388 ts_tablet_manager.cc:1428] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:55:52.786026 8491 ts_tablet_manager.cc:1918] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:52.786234 8491 log.cc:1196] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/b23a88281c4f4cd3ace2d3b1845cc7b3
I20260811 06:55:52.786875 8491 ts_tablet_manager.cc:1939] T b23a88281c4f4cd3ace2d3b1845cc7b3 P 89f61caf65d5416abeca163a013a4097: Deleting consensus metadata
I20260811 06:55:52.787256 8391 tablet_bootstrap.cc:492] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: No bootstrap required, opened a new log
I20260811 06:55:52.787472 8391 ts_tablet_manager.cc:1397] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.002s
I20260811 06:55:52.788375 8391 raft_consensus.cc:348] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.788733 8391 raft_consensus.cc:374] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.788929 7445 catalog_manager.cc:4841] TS 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239): tablet b23a88281c4f4cd3ace2d3b1845cc7b3 (table test-table [id=bfce81f57ab14468a1c7d717ea7a5a60]) successfully deleted
I20260811 06:55:52.789222 8391 raft_consensus.cc:729] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.789619 8489 tablet_replica.cc:330] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: stopping tablet replica
I20260811 06:55:52.789593 8391 consensus_queue.cc:260] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.790216 8489 raft_consensus.cc:2229] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:52.790617 8489 raft_consensus.cc:2258] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:52.792512 8389 tablet_bootstrap.cc:492] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: No bootstrap required, opened a new log
I20260811 06:55:52.792716 8389 ts_tablet_manager.cc:1397] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.010s
I20260811 06:55:52.793598 8391 ts_tablet_manager.cc:1428] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 06:55:52.793561 8389 raft_consensus.cc:348] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.793816 8389 raft_consensus.cc:374] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:52.793888 8389 raft_consensus.cc:729] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1223f3fa0b4f2384de9e88c67fd8c5, State: Initialized, Role: FOLLOWER
I20260811 06:55:52.794155 8389 consensus_queue.cc:260] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.796314 8389 ts_tablet_manager.cc:1428] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 06:55:52.804677 8489 ts_tablet_manager.cc:1905] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:52.813130 8489 ts_tablet_manager.cc:1918] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:52.813330 8489 log.cc:1196] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/b23a88281c4f4cd3ace2d3b1845cc7b3
I20260811 06:55:52.814014 8489 ts_tablet_manager.cc:1939] T b23a88281c4f4cd3ace2d3b1845cc7b3 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting consensus metadata
I20260811 06:55:52.816021 7442 catalog_manager.cc:4841] TS ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921): tablet b23a88281c4f4cd3ace2d3b1845cc7b3 (table test-table [id=bfce81f57ab14468a1c7d717ea7a5a60]) successfully deleted
I20260811 06:55:52.920221 8395 raft_consensus.cc:482] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:52.920512 8395 raft_consensus.cc:504] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.922386 8395 leader_election.cc:288] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921), 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239)
I20260811 06:55:52.941020 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.923295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:33829 (local address 127.4.40.1:40921)
0811 06:55:52.923498 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.923505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.932814 (+ 9309us) server_negotiation.cc:407] Connection header received
0811 06:55:52.933148 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.933155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.933239 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.933303 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:52.934240 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.935480 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.938262 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.939663 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.939767 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.939891 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.940511 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.940656 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:55:52.941183 8421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.923452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:35379 (local address 127.4.40.3:44239)
0811 06:55:52.923673 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:52.923680 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:52.933038 (+ 9358us) server_negotiation.cc:407] Connection header received
0811 06:55:52.933341 (+ 303us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:52.933347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:52.933435 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:52.933518 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:55:52.934643 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.935804 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.938464 (+ 2660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.939825 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.939931 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.940044 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:52.940733 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 06:55:52.940896 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 06:55:52.941862 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.922897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:40921 (local address 127.4.40.2:33829)
0811 06:55:52.932391 (+ 9494us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.932722 (+ 331us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.932952 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.933579 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.933585 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.933613 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:52.933984 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.933996 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.935713 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.935720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.938040 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.938058 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.940124 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.940131 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.940314 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.940349 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.941315 (+ 966us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.941465 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9342,"thread_start_us":9574,"threads_started":1}
I20260811 06:55:52.942492 8496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:52.923035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44239 (local address 127.4.40.2:35379)
0811 06:55:52.932919 (+ 9884us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:52.932945 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:52.933158 (+ 213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:52.933864 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:52.933874 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:52.933905 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:55:52.934434 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.934449 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.936038 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.936043 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.938246 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:52.938268 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:52.940298 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:52.940306 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:52.940509 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:52.940555 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 06:55:52.941787 (+ 1232us) client_negotiation.cc:240] Negotiation successful
0811 06:55:52.942090 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9719,"thread_start_us":9060,"threads_started":1}
I20260811 06:55:52.942577 7756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fae005747f04f2cb5282135bbc60ec0" candidate_uuid: "5e06c69d9315418990ed5669643694f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" is_pre_election: true
I20260811 06:55:52.942901 7756 raft_consensus.cc:2454] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e06c69d9315418990ed5669643694f7 in term 0.
I20260811 06:55:52.943506 7890 leader_election.cc:302] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, ef1223f3fa0b4f2384de9e88c67fd8c5; no voters:
I20260811 06:55:52.943673 8272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fae005747f04f2cb5282135bbc60ec0" candidate_uuid: "5e06c69d9315418990ed5669643694f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f61caf65d5416abeca163a013a4097" is_pre_election: true
I20260811 06:55:52.943943 8395 raft_consensus.cc:2781] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:52.943957 8272 raft_consensus.cc:2454] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e06c69d9315418990ed5669643694f7 in term 0.
I20260811 06:55:52.944073 8395 raft_consensus.cc:482] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:52.944177 8395 raft_consensus.cc:3037] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.947439 8395 raft_consensus.cc:504] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.948663 8272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fae005747f04f2cb5282135bbc60ec0" candidate_uuid: "5e06c69d9315418990ed5669643694f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89f61caf65d5416abeca163a013a4097"
I20260811 06:55:52.948868 8272 raft_consensus.cc:3037] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.948730 7756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6fae005747f04f2cb5282135bbc60ec0" candidate_uuid: "5e06c69d9315418990ed5669643694f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
I20260811 06:55:52.950306 7756 raft_consensus.cc:3037] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:52.952304 8272 raft_consensus.cc:2454] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e06c69d9315418990ed5669643694f7 in term 1.
I20260811 06:55:52.952762 7896 leader_election.cc:302] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, 89f61caf65d5416abeca163a013a4097; no voters:
I20260811 06:55:52.953358 8395 leader_election.cc:288] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [CANDIDATE]: Term 1 election: Requested vote from peers ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921), 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239)
I20260811 06:55:52.953547 8395 raft_consensus.cc:2781] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:52.953685 8395 raft_consensus.cc:686] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 1 LEADER]: Becoming Leader. State: Replica: 5e06c69d9315418990ed5669643694f7, State: Running, Role: LEADER
I20260811 06:55:52.954089 7756 raft_consensus.cc:2454] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e06c69d9315418990ed5669643694f7 in term 1.
I20260811 06:55:52.954113 8395 consensus_queue.cc:237] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:52.968045 7465 catalog_manager.cc:5495] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e06c69d9315418990ed5669643694f7 (127.4.40.2). New cstate: current_term: 1 leader_uuid: "5e06c69d9315418990ed5669643694f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } health_report { overall_health: UNKNOWN } } }
I20260811 06:55:53.039778 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:53.027261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.4.40.2:33759)
0811 06:55:53.032317 (+ 5056us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:53.032323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:53.032344 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:55:53.032443 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:53.032448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:53.032596 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:53.032767 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:53.033312 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.034753 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.036854 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.037603 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.037750 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:53.038232 (+ 482us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:53.038991 (+ 759us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:53.039170 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:53.039296 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 06:55:53.039402 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4899,"thread_start_us":4763,"threads_started":1}
I20260811 06:55:53.059067 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:53.051794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:40921 (local address 127.4.40.2:39967)
0811 06:55:53.052555 (+ 761us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:53.052576 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:53.052767 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:53.053243 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:53.053248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:53.053261 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:53.053575 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:53.053585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.055142 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.055147 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:53.056787 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:53.056804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.057894 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.057899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:53.058029 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:53.058097 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:53.058588 (+ 491us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:53.058596 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:55:53.058657 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 06:55:53.058755 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"spinlock_wait_cycles":7936,"thread_start_us":530,"threads_started":1}
I20260811 06:55:53.059144 8386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:53.052158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:39967 (local address 127.4.40.1:40921)
0811 06:55:53.052323 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:53.052328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:53.052693 (+ 365us) server_negotiation.cc:407] Connection header received
0811 06:55:53.052910 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:53.052916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:53.053000 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:53.053087 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:53.053772 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.054987 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.056953 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.057707 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.057860 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:53.058306 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:53.058481 (+ 175us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:53.058581 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:53.058716 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 06:55:53.058810 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:55:53.068073 8272 raft_consensus.cc:1261] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Refusing update from remote peer 5e06c69d9315418990ed5669643694f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:53.068183 7756 raft_consensus.cc:1261] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Refusing update from remote peer 5e06c69d9315418990ed5669643694f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:53.068845 8498 consensus_queue.cc:1035] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:53.069554 8395 consensus_queue.cc:1035] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:53.105234 8500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:53.098615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:51973 (local address 127.4.40.2:33759)
0811 06:55:53.099080 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:53.099086 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:53.099123 (+ 37us) server_negotiation.cc:407] Connection header received
0811 06:55:53.099191 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:53.099196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:53.099286 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:53.099407 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:53.099851 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.101128 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.103369 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.103774 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.103871 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:53.104439 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:53.104540 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:53.104608 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:53.104830 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 06:55:53.104908 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350}
I20260811 06:55:53.105262 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:53.098250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:33759 (local address 127.4.40.1:51973)
0811 06:55:53.098705 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 06:55:53.098727 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:55:53.098857 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:55:53.099450 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:55:53.099457 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:55:53.099471 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:55:53.099691 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:53.099701 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.101325 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.101331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:53.103200 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:55:53.103223 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:55:53.103951 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:55:53.103958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:55:53.104211 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:53.104269 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:55:53.104665 (+ 396us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:55:53.104680 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 06:55:53.104775 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:55:53.104935 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292}
W20260811 06:55:53.106834 8446 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:55:54.724198 7465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:54.724409 7465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42834:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:54.728190 7465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Sending DeleteTablet for 3 replicas of tablet 6fae005747f04f2cb5282135bbc60ec0
I20260811 06:55:54.728943 7722 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fae005747f04f2cb5282135bbc60ec0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34400
I20260811 06:55:54.728971 7974 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fae005747f04f2cb5282135bbc60ec0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56042
I20260811 06:55:54.728995 8236 tablet_service.cc:1514] Processing DeleteTablet for tablet 6fae005747f04f2cb5282135bbc60ec0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:46650
I20260811 06:55:54.741597 8559 tablet_replica.cc:330] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: stopping tablet replica
I20260811 06:55:54.742054 8559 raft_consensus.cc:2229] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:55:54.742870 8559 raft_consensus.cc:2258] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:54.744145 8556 tablet_replica.cc:330] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: stopping tablet replica
I20260811 06:55:54.744386 8556 raft_consensus.cc:2229] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:54.744472 8559 ts_tablet_manager.cc:1905] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:54.744647 8556 raft_consensus.cc:2258] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:54.745832 8556 ts_tablet_manager.cc:1905] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:54.745904 8558 tablet_replica.cc:330] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: stopping tablet replica
I20260811 06:55:54.746093 8558 raft_consensus.cc:2229] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:54.746351 8558 raft_consensus.cc:2258] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:54.747375 8558 ts_tablet_manager.cc:1905] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:54.752539 8556 ts_tablet_manager.cc:1918] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:55:54.752691 8556 log.cc:1196] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/6fae005747f04f2cb5282135bbc60ec0
I20260811 06:55:54.753085 8559 ts_tablet_manager.cc:1918] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:55:54.753208 8556 ts_tablet_manager.cc:1939] T 6fae005747f04f2cb5282135bbc60ec0 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting consensus metadata
I20260811 06:55:54.753227 8559 log.cc:1196] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/6fae005747f04f2cb5282135bbc60ec0
I20260811 06:55:54.753669 8559 ts_tablet_manager.cc:1939] T 6fae005747f04f2cb5282135bbc60ec0 P 5e06c69d9315418990ed5669643694f7: Deleting consensus metadata
I20260811 06:55:54.754413 7442 catalog_manager.cc:4841] TS ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921): tablet 6fae005747f04f2cb5282135bbc60ec0 (table test-table [id=cbce064e321c4538ada3e8c9e5af62ee]) successfully deleted
I20260811 06:55:54.754771 8558 ts_tablet_manager.cc:1918] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:55:54.754874 7442 catalog_manager.cc:4841] TS 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759): tablet 6fae005747f04f2cb5282135bbc60ec0 (table test-table [id=cbce064e321c4538ada3e8c9e5af62ee]) successfully deleted
I20260811 06:55:54.754923 8558 log.cc:1196] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/6fae005747f04f2cb5282135bbc60ec0
I20260811 06:55:54.755347 8558 ts_tablet_manager.cc:1939] T 6fae005747f04f2cb5282135bbc60ec0 P 89f61caf65d5416abeca163a013a4097: Deleting consensus metadata
I20260811 06:55:54.756261 7445 catalog_manager.cc:4841] TS 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239): tablet 6fae005747f04f2cb5282135bbc60ec0 (table test-table [id=cbce064e321c4538ada3e8c9e5af62ee]) successfully deleted
Aug 11 06:55:54 dist-test-slave-7vjv krb5kdc[7387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:55:54 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:54.795791 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:54.781146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42858 (local address 127.4.40.62:35193)
0811 06:55:54.781740 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:54.781746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:54.781852 (+ 106us) server_negotiation.cc:407] Connection header received
0811 06:55:54.782075 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:54.782079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:54.782144 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:54.782254 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:54.783588 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:54.784463 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:54.785751 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:54.786144 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:54.786303 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:54.790240 (+ 3937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:54.790246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:54.790249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:54.790279 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:54.793171 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:54.793798 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:54.793803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:54.793805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:54.793941 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:54.794338 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:54.794343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:54.794348 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:54.794620 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:54.794894 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:54.795220 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 06:55:54.795446 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":279,"threads_started":1}
I20260811 06:55:54.799453 7465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:54.800196 7465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:54.806869 7722 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec178dd106f402e96751ef274371e96 (DEFAULT_TABLE table=test-table [id=8fc86902644c43609c7148ae6a9327a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:54.807281 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec178dd106f402e96751ef274371e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:54.807253 8236 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec178dd106f402e96751ef274371e96 (DEFAULT_TABLE table=test-table [id=8fc86902644c43609c7148ae6a9327a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:54.807228 7974 tablet_service.cc:1467] Processing CreateTablet for tablet 8ec178dd106f402e96751ef274371e96 (DEFAULT_TABLE table=test-table [id=8fc86902644c43609c7148ae6a9327a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:54.807662 8236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec178dd106f402e96751ef274371e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:54.807683 7974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ec178dd106f402e96751ef274371e96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:54.817669 8570 tablet_bootstrap.cc:492] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Bootstrap starting.
I20260811 06:55:54.818966 8569 tablet_bootstrap.cc:492] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Bootstrap starting.
I20260811 06:55:54.820135 8570 tablet_bootstrap.cc:654] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:54.820981 8569 tablet_bootstrap.cc:654] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:54.822291 8571 tablet_bootstrap.cc:492] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Bootstrap starting.
I20260811 06:55:54.823673 8569 tablet_bootstrap.cc:492] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: No bootstrap required, opened a new log
I20260811 06:55:54.823805 8571 tablet_bootstrap.cc:654] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:54.823982 8569 ts_tablet_manager.cc:1397] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 06:55:54.825490 8569 raft_consensus.cc:348] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.825814 8569 raft_consensus.cc:374] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:54.825930 8569 raft_consensus.cc:729] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1223f3fa0b4f2384de9e88c67fd8c5, State: Initialized, Role: FOLLOWER
I20260811 06:55:54.826251 8569 consensus_queue.cc:260] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.827039 8569 ts_tablet_manager.cc:1428] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:55:54.828547 8571 tablet_bootstrap.cc:492] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: No bootstrap required, opened a new log
I20260811 06:55:54.828706 8571 ts_tablet_manager.cc:1397] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 06:55:54.829463 8571 raft_consensus.cc:348] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.829711 8571 raft_consensus.cc:374] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:54.829851 8571 raft_consensus.cc:729] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e06c69d9315418990ed5669643694f7, State: Initialized, Role: FOLLOWER
I20260811 06:55:54.830124 8571 consensus_queue.cc:260] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.830653 8571 ts_tablet_manager.cc:1428] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:55:54.831436 8570 tablet_bootstrap.cc:492] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: No bootstrap required, opened a new log
I20260811 06:55:54.831588 8570 ts_tablet_manager.cc:1397] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20260811 06:55:54.832345 8570 raft_consensus.cc:348] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.832540 8570 raft_consensus.cc:374] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:54.832672 8570 raft_consensus.cc:729] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Initialized, Role: FOLLOWER
I20260811 06:55:54.832940 8570 consensus_queue.cc:260] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.833844 8570 ts_tablet_manager.cc:1428] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 06:55:54.841180 8576 raft_consensus.cc:482] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:54.841328 8576 raft_consensus.cc:504] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.841913 8576 leader_election.cc:288] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921), 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759)
I20260811 06:55:54.842442 7756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec178dd106f402e96751ef274371e96" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" is_pre_election: true
I20260811 06:55:54.842512 8022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec178dd106f402e96751ef274371e96" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7" is_pre_election: true
I20260811 06:55:54.842665 7756 raft_consensus.cc:2454] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 0.
I20260811 06:55:54.842804 8022 raft_consensus.cc:2454] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 0.
I20260811 06:55:54.843204 8136 leader_election.cc:302] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89f61caf65d5416abeca163a013a4097, ef1223f3fa0b4f2384de9e88c67fd8c5; no voters:
I20260811 06:55:54.843513 8576 raft_consensus.cc:2781] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:54.843640 8576 raft_consensus.cc:482] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:54.843741 8576 raft_consensus.cc:3037] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:54.845784 8576 raft_consensus.cc:504] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.846282 8576 leader_election.cc:288] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 election: Requested vote from peers ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921), 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759)
I20260811 06:55:54.846755 7756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec178dd106f402e96751ef274371e96" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
I20260811 06:55:54.846776 8022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ec178dd106f402e96751ef274371e96" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7"
I20260811 06:55:54.846948 8022 raft_consensus.cc:3037] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:54.846948 7756 raft_consensus.cc:3037] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:54.849954 8022 raft_consensus.cc:2454] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 1.
I20260811 06:55:54.849954 7756 raft_consensus.cc:2454] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 1.
I20260811 06:55:54.850344 8136 leader_election.cc:302] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e06c69d9315418990ed5669643694f7, 89f61caf65d5416abeca163a013a4097; no voters:
I20260811 06:55:54.850647 8576 raft_consensus.cc:2781] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:54.851034 8576 raft_consensus.cc:686] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 1 LEADER]: Becoming Leader. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Running, Role: LEADER
I20260811 06:55:54.851301 8576 consensus_queue.cc:237] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.853655 7465 catalog_manager.cc:5495] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f61caf65d5416abeca163a013a4097 (127.4.40.3). New cstate: current_term: 1 leader_uuid: "89f61caf65d5416abeca163a013a4097" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } health_report { overall_health: HEALTHY } } }
I20260811 06:55:54.889484 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:54.878865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46672 (local address 127.4.40.3:44239)
0811 06:55:54.884609 (+ 5744us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:54.884613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:54.884633 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:55:54.884683 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:54.884687 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:54.884750 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:54.884868 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:55:54.885307 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:54.886204 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:54.887504 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:54.887949 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:54.888120 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:54.888621 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:55:54.888736 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:55:54.888860 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:54.889035 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 06:55:54.889130 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5620,"thread_start_us":5465,"threads_started":1}
I20260811 06:55:54.898604 8022 raft_consensus.cc:1261] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Refusing update from remote peer 89f61caf65d5416abeca163a013a4097: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:54.898717 7756 raft_consensus.cc:1261] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Refusing update from remote peer 89f61caf65d5416abeca163a013a4097: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:54.899216 8576 consensus_queue.cc:1035] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:54.899480 8577 consensus_queue.cc:1035] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:54.947876 7465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:54.948182 7465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:54.950995 7465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Sending DeleteTablet for 3 replicas of tablet 8ec178dd106f402e96751ef274371e96
I20260811 06:55:54.951736 7722 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec178dd106f402e96751ef274371e96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34400
I20260811 06:55:54.951833 7974 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec178dd106f402e96751ef274371e96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56042
I20260811 06:55:54.951891 8236 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ec178dd106f402e96751ef274371e96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:46650
I20260811 06:55:54.952020 8556 tablet_replica.cc:330] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: stopping tablet replica
I20260811 06:55:54.952225 8558 tablet_replica.cc:330] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: stopping tablet replica
I20260811 06:55:54.952289 8559 tablet_replica.cc:330] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: stopping tablet replica
I20260811 06:55:54.952272 8556 raft_consensus.cc:2229] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:54.952442 8558 raft_consensus.cc:2229] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:55:54.952483 8559 raft_consensus.cc:2229] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:54.952468 8556 raft_consensus.cc:2258] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:54.952664 8559 raft_consensus.cc:2258] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:54.952786 8558 raft_consensus.cc:2258] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:54.953065 7465 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:55:54.953715 7465 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:55:54.953850 8559 ts_tablet_manager.cc:1905] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:54.954068 8556 ts_tablet_manager.cc:1905] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:54.960006 7722 tablet_service.cc:1467] Processing CreateTablet for tablet 230a854abd1f4a79b33e4dcfdfa50c34 (DEFAULT_TABLE table=test-table [id=8facba6d57d14702ab9a672e37e59fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:54.960132 7974 tablet_service.cc:1467] Processing CreateTablet for tablet 230a854abd1f4a79b33e4dcfdfa50c34 (DEFAULT_TABLE table=test-table [id=8facba6d57d14702ab9a672e37e59fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:54.960279 8236 tablet_service.cc:1467] Processing CreateTablet for tablet 230a854abd1f4a79b33e4dcfdfa50c34 (DEFAULT_TABLE table=test-table [id=8facba6d57d14702ab9a672e37e59fa1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:55:54.960476 7722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 230a854abd1f4a79b33e4dcfdfa50c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:54.960536 7974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 230a854abd1f4a79b33e4dcfdfa50c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:54.960566 8559 ts_tablet_manager.cc:1918] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:54.960687 8236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 230a854abd1f4a79b33e4dcfdfa50c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:55:54.960695 8559 log.cc:1196] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/8ec178dd106f402e96751ef274371e96
I20260811 06:55:54.961156 8559 ts_tablet_manager.cc:1939] T 8ec178dd106f402e96751ef274371e96 P 5e06c69d9315418990ed5669643694f7: Deleting consensus metadata
I20260811 06:55:54.965651 8571 tablet_bootstrap.cc:492] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Bootstrap starting.
I20260811 06:55:54.965930 7442 catalog_manager.cc:4841] TS 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759): tablet 8ec178dd106f402e96751ef274371e96 (table test-table [id=8fc86902644c43609c7148ae6a9327a6]) successfully deleted
I20260811 06:55:54.966604 8569 tablet_bootstrap.cc:492] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Bootstrap starting.
I20260811 06:55:54.966895 8558 ts_tablet_manager.cc:1905] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:54.967330 8571 tablet_bootstrap.cc:654] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:54.968004 8569 tablet_bootstrap.cc:654] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:54.969606 8571 tablet_bootstrap.cc:492] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: No bootstrap required, opened a new log
I20260811 06:55:54.969755 8571 ts_tablet_manager.cc:1397] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 06:55:54.970268 8569 tablet_bootstrap.cc:492] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: No bootstrap required, opened a new log
I20260811 06:55:54.970459 8569 ts_tablet_manager.cc:1397] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 06:55:54.970532 8571 raft_consensus.cc:348] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.970781 8571 raft_consensus.cc:374] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:54.970852 8571 raft_consensus.cc:729] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e06c69d9315418990ed5669643694f7, State: Initialized, Role: FOLLOWER
I20260811 06:55:54.971102 8571 consensus_queue.cc:260] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.971237 8569 raft_consensus.cc:348] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.971488 8569 raft_consensus.cc:374] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:54.971583 8569 raft_consensus.cc:729] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef1223f3fa0b4f2384de9e88c67fd8c5, State: Initialized, Role: FOLLOWER
I20260811 06:55:54.971853 8569 consensus_queue.cc:260] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.972250 8571 ts_tablet_manager.cc:1428] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260811 06:55:54.972433 8569 ts_tablet_manager.cc:1428] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
I20260811 06:55:54.973095 8570 tablet_bootstrap.cc:492] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Bootstrap starting.
I20260811 06:55:54.973522 8558 ts_tablet_manager.cc:1918] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:54.973678 8558 log.cc:1196] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/8ec178dd106f402e96751ef274371e96
I20260811 06:55:54.974231 8558 ts_tablet_manager.cc:1939] T 8ec178dd106f402e96751ef274371e96 P 89f61caf65d5416abeca163a013a4097: Deleting consensus metadata
I20260811 06:55:54.974920 8570 tablet_bootstrap.cc:654] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Neither blocks nor log segments found. Creating new log.
I20260811 06:55:54.975536 7445 catalog_manager.cc:4841] TS 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239): tablet 8ec178dd106f402e96751ef274371e96 (table test-table [id=8fc86902644c43609c7148ae6a9327a6]) successfully deleted
I20260811 06:55:54.977010 8556 ts_tablet_manager.cc:1918] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:55:54.977175 8556 log.cc:1196] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/8ec178dd106f402e96751ef274371e96
I20260811 06:55:54.977557 8570 tablet_bootstrap.cc:492] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: No bootstrap required, opened a new log
I20260811 06:55:54.977748 8556 ts_tablet_manager.cc:1939] T 8ec178dd106f402e96751ef274371e96 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting consensus metadata
I20260811 06:55:54.977794 8570 ts_tablet_manager.cc:1397] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 06:55:54.978606 8570 raft_consensus.cc:348] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.978813 7442 catalog_manager.cc:4841] TS ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921): tablet 8ec178dd106f402e96751ef274371e96 (table test-table [id=8fc86902644c43609c7148ae6a9327a6]) successfully deleted
I20260811 06:55:54.978902 8570 raft_consensus.cc:374] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:55:54.979005 8570 raft_consensus.cc:729] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Initialized, Role: FOLLOWER
I20260811 06:55:54.979290 8570 consensus_queue.cc:260] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.979785 8570 ts_tablet_manager.cc:1428] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:55:54.985548 8590 raft_consensus.cc:482] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:55:54.985711 8590 raft_consensus.cc:504] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.986282 8590 leader_election.cc:288] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921), 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759)
I20260811 06:55:54.986778 7756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230a854abd1f4a79b33e4dcfdfa50c34" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" is_pre_election: true
I20260811 06:55:54.986835 8022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230a854abd1f4a79b33e4dcfdfa50c34" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7" is_pre_election: true
I20260811 06:55:54.987049 7756 raft_consensus.cc:2454] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 0.
I20260811 06:55:54.987047 8022 raft_consensus.cc:2454] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 0.
I20260811 06:55:54.987550 8136 leader_election.cc:302] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89f61caf65d5416abeca163a013a4097, ef1223f3fa0b4f2384de9e88c67fd8c5; no voters:
I20260811 06:55:54.987807 8590 raft_consensus.cc:2781] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:55:54.987905 8590 raft_consensus.cc:482] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:55:54.988021 8590 raft_consensus.cc:3037] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:54.989926 8590 raft_consensus.cc:504] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.990406 8590 leader_election.cc:288] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 election: Requested vote from peers ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921), 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759)
I20260811 06:55:54.990816 7756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230a854abd1f4a79b33e4dcfdfa50c34" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5"
I20260811 06:55:54.991034 7756 raft_consensus.cc:3037] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:54.991231 8022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "230a854abd1f4a79b33e4dcfdfa50c34" candidate_uuid: "89f61caf65d5416abeca163a013a4097" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5e06c69d9315418990ed5669643694f7"
I20260811 06:55:54.991458 8022 raft_consensus.cc:3037] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:55:54.993938 7756 raft_consensus.cc:2454] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 1.
I20260811 06:55:54.994333 8136 leader_election.cc:302] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89f61caf65d5416abeca163a013a4097, ef1223f3fa0b4f2384de9e88c67fd8c5; no voters:
I20260811 06:55:54.994431 8022 raft_consensus.cc:2454] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89f61caf65d5416abeca163a013a4097 in term 1.
I20260811 06:55:54.994643 8590 raft_consensus.cc:2781] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:55:54.994822 8590 raft_consensus.cc:686] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 1 LEADER]: Becoming Leader. State: Replica: 89f61caf65d5416abeca163a013a4097, State: Running, Role: LEADER
I20260811 06:55:54.995075 8590 consensus_queue.cc:237] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } }
I20260811 06:55:54.997299 7465 catalog_manager.cc:5495] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 reported cstate change: term changed from 0 to 1, leader changed from <none> to 89f61caf65d5416abeca163a013a4097 (127.4.40.3). New cstate: current_term: 1 leader_uuid: "89f61caf65d5416abeca163a013a4097" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89f61caf65d5416abeca163a013a4097" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44239 } health_report { overall_health: HEALTHY } } }
I20260811 06:55:55.019013 8024 raft_consensus.cc:1261] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Refusing update from remote peer 89f61caf65d5416abeca163a013a4097: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:55.019172 7756 raft_consensus.cc:1261] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Refusing update from remote peer 89f61caf65d5416abeca163a013a4097: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:55:55.019562 8590 consensus_queue.cc:1035] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5e06c69d9315418990ed5669643694f7" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:55:55.019850 8589 consensus_queue.cc:1035] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef1223f3fa0b4f2384de9e88c67fd8c5" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 40921 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 06:55:55.038848 8528 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 06:55:56.620539 7466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:56.620808 7466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:55:56.624611 7466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 94bee6203b364a37b1a6c1e2652ed4a1: Sending DeleteTablet for 3 replicas of tablet 230a854abd1f4a79b33e4dcfdfa50c34
I20260811 06:55:56.625569 7722 tablet_service.cc:1514] Processing DeleteTablet for tablet 230a854abd1f4a79b33e4dcfdfa50c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:56 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34400
I20260811 06:55:56.625619 8236 tablet_service.cc:1514] Processing DeleteTablet for tablet 230a854abd1f4a79b33e4dcfdfa50c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:56 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:46650
I20260811 06:55:56.625622 7974 tablet_service.cc:1514] Processing DeleteTablet for tablet 230a854abd1f4a79b33e4dcfdfa50c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:55:56 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56042
I20260811 06:55:56.626189 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7608
I20260811 06:55:56.638135 8664 tablet_replica.cc:330] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: stopping tablet replica
I20260811 06:55:56.638597 8664 raft_consensus.cc:2229] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:56.639048 8664 raft_consensus.cc:2258] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:56.639788 8668 tablet_replica.cc:330] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: stopping tablet replica
I20260811 06:55:56.640049 8668 raft_consensus.cc:2229] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:55:56.640348 8664 ts_tablet_manager.cc:1905] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:56.640380 8668 raft_consensus.cc:2258] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:56.642138 8668 ts_tablet_manager.cc:1905] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:56.647951 8664 ts_tablet_manager.cc:1918] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:55:56.648160 8670 tablet_replica.cc:330] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: stopping tablet replica
I20260811 06:55:56.648175 8664 log.cc:1196] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/230a854abd1f4a79b33e4dcfdfa50c34
I20260811 06:55:56.648458 8670 raft_consensus.cc:2229] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:55:56.648702 8664 ts_tablet_manager.cc:1939] T 230a854abd1f4a79b33e4dcfdfa50c34 P ef1223f3fa0b4f2384de9e88c67fd8c5: Deleting consensus metadata
I20260811 06:55:56.648931 8670 raft_consensus.cc:2258] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:55:56.650028 7442 catalog_manager.cc:4841] TS ef1223f3fa0b4f2384de9e88c67fd8c5 (127.4.40.1:40921): tablet 230a854abd1f4a79b33e4dcfdfa50c34 (table test-table [id=8facba6d57d14702ab9a672e37e59fa1]) successfully deleted
I20260811 06:55:56.650080 8670 ts_tablet_manager.cc:1905] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:55:56.650281 8668 ts_tablet_manager.cc:1918] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:55:56.650449 8668 log.cc:1196] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/230a854abd1f4a79b33e4dcfdfa50c34
I20260811 06:55:56.650879 8668 ts_tablet_manager.cc:1939] T 230a854abd1f4a79b33e4dcfdfa50c34 P 5e06c69d9315418990ed5669643694f7: Deleting consensus metadata
I20260811 06:55:56.651780 7442 catalog_manager.cc:4841] TS 5e06c69d9315418990ed5669643694f7 (127.4.40.2:33759): tablet 230a854abd1f4a79b33e4dcfdfa50c34 (table test-table [id=8facba6d57d14702ab9a672e37e59fa1]) successfully deleted
I20260811 06:55:56.654891 8670 ts_tablet_manager.cc:1918] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 06:55:56.655042 8670 log.cc:1196] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/230a854abd1f4a79b33e4dcfdfa50c34
I20260811 06:55:56.655417 8670 ts_tablet_manager.cc:1939] T 230a854abd1f4a79b33e4dcfdfa50c34 P 89f61caf65d5416abeca163a013a4097: Deleting consensus metadata
Aug 11 06:55:56 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:55:56.656424 7445 catalog_manager.cc:4841] TS 89f61caf65d5416abeca163a013a4097 (127.4.40.3:44239): tablet 230a854abd1f4a79b33e4dcfdfa50c34 (table test-table [id=8facba6d57d14702ab9a672e37e59fa1]) successfully deleted
I20260811 06:55:56.661936 8673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:56.627715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.4.40.1:40921)
0811 06:55:56.641624 (+ 13909us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:56.641630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:56.641654 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:55:56.641706 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:56.641711 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:56.641805 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:56.641907 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:56.643892 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:56.645020 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:56.646801 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:56.647394 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:56.647532 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:56.656754 (+ 9222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:56.656770 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:56.656787 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:56.656818 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:56.659591 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:56.660269 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:56.660276 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:56.660279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:56.660410 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:56.660733 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:56.660736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:56.660738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:56.661093 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:56.661314 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:56.661527 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 06:55:56.661681 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13751,"thread_start_us":13664,"threads_started":1}
W20260811 06:55:56.662772 7849 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34414
W20260811 06:55:56.663419 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7608: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:55:56.663653 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7608
I20260811 06:55:56.700314 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7862
Aug 11 06:55:56 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:55:56.714990 8674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:56.701034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56050 (local address 127.4.40.2:33759)
0811 06:55:56.701535 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:56.701540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:56.701556 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:55:56.701611 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:56.701615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:56.701685 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:56.701775 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:56.702902 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:56.703662 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:56.704953 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:56.705351 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:56.705444 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:56.709434 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:56.709447 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:56.709462 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:56.709496 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:56.712534 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:56.713127 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:56.713134 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:56.713136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:56.713291 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:56.713594 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:56.713597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:56.713599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:56.713931 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:56.714162 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:56.714418 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 06:55:56.714601 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":236,"threads_started":1}
W20260811 06:55:56.715672 8096 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56050
W20260811 06:55:56.716416 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7862: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:55:56.716583 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7862
I20260811 06:55:56.750536 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8112
Aug 11 06:55:56 dist-test-slave-7vjv krb5kdc[7387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431354, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:55:56.765412 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:56.751339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46688 (local address 127.4.40.3:44239)
0811 06:55:56.751873 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:56.751877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:56.751893 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:55:56.751951 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:56.751954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:56.752021 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:56.752169 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:56.753484 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:56.754226 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:56.755370 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:56.755792 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:56.755881 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:56.760032 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:56.760046 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:56.760062 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:56.760147 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:55:56.762864 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:56.763483 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:56.763490 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:56.763493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:56.763618 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:55:56.763928 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:55:56.763931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:55:56.763933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:55:56.764372 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:55:56.764646 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 06:55:56.764923 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:55:56.765093 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":295,"threads_started":1}
W20260811 06:55:56.766134 8350 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46688
W20260811 06:55:56.766748 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8112: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:55:56.766961 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8112
I20260811 06:55:56.800516 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7415
I20260811 06:55:56.801741 7553 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:55:56.918383 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7415
2026-08-11T06:55:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8805 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:55:57 dist-test-slave-7vjv krb5kdc[8688](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:55:57 dist-test-slave-7vjv krb5kdc[8688](info): set up 2 sockets
Aug 11 06:55:57 dist-test-slave-7vjv krb5kdc[8688](info): commencing operation
krb5kdc: starting...
W20260811 06:55:58.932698 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.958s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:55:59 dist-test-slave-7vjv krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:55:59.296319 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45317
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:55:59.855504 8712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:55:59.855772 8712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:55:59.855947 8712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:55:59.863046 8712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:55:59.863171 8712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:55:59.863263 8712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:55:59.863382 8712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:55:59.863461 8712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:55:59.873124 8712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:55:59.874348 8712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:55:59.875689 8712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:55:59.887977 8721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:59.888024 8722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:55:59.888075 8726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:55:59.888654 8712 server_base.cc:1034] running on GCE node
I20260811 06:55:59.889436 8712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:55:59.890785 8712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:55:59.892022 8712 hybrid_clock.cc:648] HybridClock initialized: now 1786431359891980 us; error 29 us; skew 500 ppm
I20260811 06:55:59.892496 8712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:55:59 dist-test-slave-7vjv krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431359, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:55:59.895613 8712 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:55:59.897132 8712 webserver.cc:460] Webserver started at http://127.4.40.62:45165/ using document root <none> and password file <none>
I20260811 06:55:59.897814 8712 fs_manager.cc:359] Metadata directory not provided
I20260811 06:55:59.897971 8712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:55:59.898294 8712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:55:59.901324 8712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "7dba89a436b7412da8387abe395663b1"
format_stamp: "Formatted at 2026-08-11 06:55:59 on dist-test-slave-7vjv"
server_key: "c32246376ddcf9f090a574fc9455f4c7"
server_key_iv: "7e25aa0738e1b4dcc93902399b396b35"
server_key_version: "encryptionkey@0"
I20260811 06:55:59.902122 8712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "7dba89a436b7412da8387abe395663b1"
format_stamp: "Formatted at 2026-08-11 06:55:59 on dist-test-slave-7vjv"
server_key: "c32246376ddcf9f090a574fc9455f4c7"
server_key_iv: "7e25aa0738e1b4dcc93902399b396b35"
server_key_version: "encryptionkey@0"
I20260811 06:55:59.907335 8712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:55:59.910843 8735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:55:59.912195 8712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:55:59.912377 8712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "7dba89a436b7412da8387abe395663b1"
format_stamp: "Formatted at 2026-08-11 06:55:59 on dist-test-slave-7vjv"
server_key: "c32246376ddcf9f090a574fc9455f4c7"
server_key_iv: "7e25aa0738e1b4dcc93902399b396b35"
server_key_version: "encryptionkey@0"
I20260811 06:55:59.912573 8712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:55:59.955856 8712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:55:59.959976 8712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:55:59.960312 8712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:55:59.979684 8712 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45317
I20260811 06:55:59.979710 8826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45317 every 8 connection(s)
I20260811 06:55:59.981091 8712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:55:59.981794 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8712
I20260811 06:55:59.982082 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:55:59.982826 4256 external_mini_cluster.cc:1428] Setting key e9086c1d47f6d3daba8f5ed6be7fdeed
I20260811 06:55:59.985671 8827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:55:59 dist-test-slave-7vjv krb5kdc[8688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:55:59.998145 8827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1: Bootstrap starting.
I20260811 06:56:00.002324 8827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:00.003638 8827 log.cc:824] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:00.006506 8827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1: No bootstrap required, opened a new log
I20260811 06:56:00.007345 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:55:59.984856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34210 (local address 127.4.40.62:45317)
0811 06:55:59.985749 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 06:55:59.985763 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:55:59.985814 (+ 51us) server_negotiation.cc:407] Connection header received
0811 06:55:59.988287 (+ 2473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:55:59.988314 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:55:59.988799 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:55:59.989236 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:55:59.990555 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:59.992458 (+ 1903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:59.993714 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:55:59.994735 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:55:59.994933 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:55:59.998901 (+ 3968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:55:59.998911 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:55:59.998927 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:55:59.998975 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:00.003115 (+ 4140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:00.003897 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:00.003904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:00.003909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:00.004080 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:00.004548 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:00.004559 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:00.004562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:00.005127 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:00.005435 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:00.005891 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 06:56:00.006310 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":638,"thread_start_us":356,"threads_started":1}
I20260811 06:56:00.015240 8827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dba89a436b7412da8387abe395663b1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45317 } }
I20260811 06:56:00.015597 8827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:00.015763 8827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dba89a436b7412da8387abe395663b1, State: Initialized, Role: FOLLOWER
I20260811 06:56:00.016381 8827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dba89a436b7412da8387abe395663b1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45317 } }
I20260811 06:56:00.016623 8827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:00.016780 8827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:00.016993 8827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:00.019297 8827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dba89a436b7412da8387abe395663b1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45317 } }
I20260811 06:56:00.019860 8827 leader_election.cc:302] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dba89a436b7412da8387abe395663b1; no voters:
I20260811 06:56:00.020581 8827 leader_election.cc:288] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:00.020711 8838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:00.021493 8838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [term 1 LEADER]: Becoming Leader. State: Replica: 7dba89a436b7412da8387abe395663b1, State: Running, Role: LEADER
I20260811 06:56:00.022063 8838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dba89a436b7412da8387abe395663b1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45317 } }
I20260811 06:56:00.022620 8827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:00.025120 8839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dba89a436b7412da8387abe395663b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dba89a436b7412da8387abe395663b1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45317 } } }
I20260811 06:56:00.025098 8841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dba89a436b7412da8387abe395663b1. Latest consensus state: current_term: 1 leader_uuid: "7dba89a436b7412da8387abe395663b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dba89a436b7412da8387abe395663b1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45317 } } }
I20260811 06:56:00.025506 8839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:00.025552 8841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:00.026994 8852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:00.031878 8852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:00.041445 8852 catalog_manager.cc:1347] Generated new cluster ID: a90e312d54fc439b848424fbc5623d52
I20260811 06:56:00.041560 8852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:00.050891 8852 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:00.052750 8852 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:00.061932 8852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dba89a436b7412da8387abe395663b1: Generated new TSK 0
I20260811 06:56:00.062866 8852 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:56:00.123387 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45317
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:00.465161 8884 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:00.465467 8884 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:00.465564 8884 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:00.472412 8884 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:00.472564 8884 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:00.472689 8884 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:00.482265 8884 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:00.483628 8884 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:00.485342 8884 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:00.498870 8900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:00.498848 8905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:00.499176 8897 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:00.499787 8884 server_base.cc:1034] running on GCE node
I20260811 06:56:00.500438 8884 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:00.501401 8884 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:00.502640 8884 hybrid_clock.cc:648] HybridClock initialized: now 1786431360502592 us; error 58 us; skew 500 ppm
I20260811 06:56:00.502964 8884 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:00 dist-test-slave-7vjv krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431360, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:00.506217 8884 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:56:00.508291 8884 webserver.cc:460] Webserver started at http://127.4.40.1:35941/ using document root <none> and password file <none>
I20260811 06:56:00.508961 8884 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:00.509107 8884 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:00.509452 8884 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:00.512580 8884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "d2d77a6cac1f4a75a9d28f72e8d59990"
format_stamp: "Formatted at 2026-08-11 06:56:00 on dist-test-slave-7vjv"
server_key: "ff15b7820218754757840c4f5faca7e8"
server_key_iv: "082863520eea5018af740a59b63f2512"
server_key_version: "encryptionkey@0"
I20260811 06:56:00.513396 8884 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "d2d77a6cac1f4a75a9d28f72e8d59990"
format_stamp: "Formatted at 2026-08-11 06:56:00 on dist-test-slave-7vjv"
server_key: "ff15b7820218754757840c4f5faca7e8"
server_key_iv: "082863520eea5018af740a59b63f2512"
server_key_version: "encryptionkey@0"
I20260811 06:56:00.518839 8884 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 06:56:00.522449 8918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:00.523953 8884 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:56:00.524199 8884 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "d2d77a6cac1f4a75a9d28f72e8d59990"
format_stamp: "Formatted at 2026-08-11 06:56:00 on dist-test-slave-7vjv"
server_key: "ff15b7820218754757840c4f5faca7e8"
server_key_iv: "082863520eea5018af740a59b63f2512"
server_key_version: "encryptionkey@0"
I20260811 06:56:00.524421 8884 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:00.553467 8884 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:00.557561 8884 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:00.557868 8884 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:00.558852 8884 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:00.560303 8884 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:00.560407 8884 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:00.560568 8884 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:00.560657 8884 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:00.595027 8884 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:43717
I20260811 06:56:00.595088 9141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:43717 every 8 connection(s)
I20260811 06:56:00.596438 8884 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:00.606472 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8884
I20260811 06:56:00.606734 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:00.607486 4256 external_mini_cluster.cc:1428] Setting key d53f9da828325f6d7dae266575868dc2
Aug 11 06:56:00 dist-test-slave-7vjv krb5kdc[8688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431360, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:00.616436 9151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:00.599004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:33689 (local address 127.4.40.62:45317)
0811 06:56:00.599454 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:00.599458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:00.600497 (+ 1039us) server_negotiation.cc:407] Connection header received
0811 06:56:00.602361 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:00.602366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:00.602433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:00.602533 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:00.603959 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:00.604740 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:00.606216 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:00.606844 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:00.606991 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:00.610609 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:00.610614 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:00.610617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:00.610647 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:00.613365 (+ 2718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:00.614068 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:00.614075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:00.614078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:00.614204 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:00.614658 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:00.614664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:00.614667 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:00.614912 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:00.615077 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:00.615799 (+ 722us) server_negotiation.cc:299] Negotiation successful
0811 06:56:00.616031 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":220,"threads_started":1}
I20260811 06:56:00.617357 9153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:00.599571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45317 (local address 127.4.40.1:33689)
0811 06:56:00.600339 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:00.600378 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:00.602113 (+ 1735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:00.602756 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:00.602763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:00.603251 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:00.603810 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:00.603826 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:00.604880 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:00.604883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:00.606070 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:00.606082 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:00.607056 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:00.607061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:00.607249 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:00.607911 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:00.607933 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:00.610388 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:00.613526 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:00.613534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:00.613547 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:00.613948 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:00.614317 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:00.614321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:00.614325 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:00.614530 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:00.615048 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:00.615055 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:00.615379 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 06:56:00.615681 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 06:56:00.616205 (+ 524us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":288,"threads_started":1}
I20260811 06:56:00.618887 9149 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45317
I20260811 06:56:00.619230 9149 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:00.619926 9149 heartbeater.cc:511] Master 127.4.40.62:45317 requested a full tablet report, sending...
I20260811 06:56:00.622196 8767 ts_manager.cc:194] Registered new tserver with Master: d2d77a6cac1f4a75a9d28f72e8d59990 (127.4.40.1:43717)
I20260811 06:56:00.624461 8767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:33689
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:56:00.662828 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45317
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:00.994138 9158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:00.994388 9158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:00.994480 9158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:01.001593 9158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:01.001690 9158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:01.001801 9158 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:01.011262 9158 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:01.012503 9158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:01.013773 9158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:01.027202 9165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:01.027144 9174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:01.027227 9166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:01.028165 9158 server_base.cc:1034] running on GCE node
I20260811 06:56:01.028828 9158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:01.029753 9158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:01.030987 9158 hybrid_clock.cc:648] HybridClock initialized: now 1786431361030946 us; error 30 us; skew 500 ppm
I20260811 06:56:01.031353 9158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:01 dist-test-slave-7vjv krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431361, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:01.034756 9158 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:56:01.036320 9158 webserver.cc:460] Webserver started at http://127.4.40.2:40349/ using document root <none> and password file <none>
I20260811 06:56:01.036998 9158 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:01.037158 9158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:01.037488 9158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:01.041355 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "7cc7a0f12e6843b7b4a37b500694021c"
format_stamp: "Formatted at 2026-08-11 06:56:01 on dist-test-slave-7vjv"
server_key: "4e700c07b518c5e594340b59c380ad63"
server_key_iv: "352c489028a6f30b392b01e53b48953f"
server_key_version: "encryptionkey@0"
I20260811 06:56:01.042424 9158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "7cc7a0f12e6843b7b4a37b500694021c"
format_stamp: "Formatted at 2026-08-11 06:56:01 on dist-test-slave-7vjv"
server_key: "4e700c07b518c5e594340b59c380ad63"
server_key_iv: "352c489028a6f30b392b01e53b48953f"
server_key_version: "encryptionkey@0"
I20260811 06:56:01.047554 9158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:56:01.051148 9185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:01.052464 9158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:01.052639 9158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "7cc7a0f12e6843b7b4a37b500694021c"
format_stamp: "Formatted at 2026-08-11 06:56:01 on dist-test-slave-7vjv"
server_key: "4e700c07b518c5e594340b59c380ad63"
server_key_iv: "352c489028a6f30b392b01e53b48953f"
server_key_version: "encryptionkey@0"
I20260811 06:56:01.052840 9158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:01.075017 9158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:01.079000 9158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:01.079308 9158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:01.080351 9158 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:01.081713 9158 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:01.081841 9158 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:01.081961 9158 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:01.082015 9158 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:01.116439 9158 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:40499
I20260811 06:56:01.116533 9389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:40499 every 8 connection(s)
I20260811 06:56:01.117810 9158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:01.122689 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9158
I20260811 06:56:01.122891 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:01.123389 4256 external_mini_cluster.cc:1428] Setting key 645a262d9f32efcfbe1e2173e9aa8749
Aug 11 06:56:01 dist-test-slave-7vjv krb5kdc[8688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431361, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:01.138721 9395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:01.120382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:48767 (local address 127.4.40.62:45317)
0811 06:56:01.120873 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:01.120877 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:01.121743 (+ 866us) server_negotiation.cc:407] Connection header received
0811 06:56:01.123900 (+ 2157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:01.123906 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:01.124006 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:01.124167 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:01.125682 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.126435 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.128132 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.128577 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.128736 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:01.132530 (+ 3794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:01.132538 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:01.132543 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:01.132589 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:01.135325 (+ 2736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:01.136287 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:01.136292 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:01.136294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:01.136453 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:01.136949 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:01.136952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:01.136954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:01.137225 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:01.137338 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:01.138256 (+ 918us) server_negotiation.cc:299] Negotiation successful
0811 06:56:01.138432 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":266,"threads_started":1}
I20260811 06:56:01.140364 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:01.120844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45317 (local address 127.4.40.2:48767)
0811 06:56:01.121581 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:01.121625 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:01.123632 (+ 2007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:01.124415 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:01.124424 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:01.124942 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:01.125505 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:01.125527 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.126602 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.126606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:01.127960 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:01.127975 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.128851 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.128861 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:01.129099 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:01.129790 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:01.129812 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:01.132276 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:01.135468 (+ 3192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:01.135479 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:01.135497 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:01.136152 (+ 655us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:01.136572 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:01.136579 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:01.136586 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:01.136838 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:01.137357 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:01.137367 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:01.137847 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 06:56:01.138187 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 06:56:01.138812 (+ 625us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":300,"threads_started":1}
I20260811 06:56:01.141925 9392 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45317
I20260811 06:56:01.142280 9392 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:01.143178 9392 heartbeater.cc:511] Master 127.4.40.62:45317 requested a full tablet report, sending...
I20260811 06:56:01.144893 8767 ts_manager.cc:194] Registered new tserver with Master: 7cc7a0f12e6843b7b4a37b500694021c (127.4.40.2:40499)
I20260811 06:56:01.145702 8767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:48767
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:56:01.179137 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45317
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:01.522603 9401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:01.522938 9401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:01.523100 9401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:01.530570 9401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:01.530722 9401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:01.530900 9401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:01.540731 9401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:01.542073 9401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:01.543387 9401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:01.556979 9416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:01.556979 9409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:01.557531 9411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:01.558161 9401 server_base.cc:1034] running on GCE node
I20260811 06:56:01.558825 9401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:01.559824 9401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:01.561079 9401 hybrid_clock.cc:648] HybridClock initialized: now 1786431361560989 us; error 68 us; skew 500 ppm
I20260811 06:56:01.561424 9401 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:01 dist-test-slave-7vjv krb5kdc[8688](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431361, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:01.564605 9401 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:56:01.565963 9401 webserver.cc:460] Webserver started at http://127.4.40.3:33517/ using document root <none> and password file <none>
I20260811 06:56:01.566609 9401 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:01.566715 9401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:01.566975 9401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:01.569909 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "4a86ce20daf94055bae6b6ca33fb9da0"
format_stamp: "Formatted at 2026-08-11 06:56:01 on dist-test-slave-7vjv"
server_key: "540a99159cddf9ad73428c18bcb4989e"
server_key_iv: "7757bfa0fe8dfe399e7ab49a2ffca21b"
server_key_version: "encryptionkey@0"
I20260811 06:56:01.570664 9401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "4a86ce20daf94055bae6b6ca33fb9da0"
format_stamp: "Formatted at 2026-08-11 06:56:01 on dist-test-slave-7vjv"
server_key: "540a99159cddf9ad73428c18bcb4989e"
server_key_iv: "7757bfa0fe8dfe399e7ab49a2ffca21b"
server_key_version: "encryptionkey@0"
I20260811 06:56:01.576236 9401 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:56:01.579594 9428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:01.580996 9401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:01.581142 9401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "4a86ce20daf94055bae6b6ca33fb9da0"
format_stamp: "Formatted at 2026-08-11 06:56:01 on dist-test-slave-7vjv"
server_key: "540a99159cddf9ad73428c18bcb4989e"
server_key_iv: "7757bfa0fe8dfe399e7ab49a2ffca21b"
server_key_version: "encryptionkey@0"
I20260811 06:56:01.581323 9401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:01.600682 9401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:01.604568 9401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:01.604809 9401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:01.605679 9401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:01.606969 9401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:01.607049 9401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:01.607144 9401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:01.607187 9401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:01.627933 9149 heartbeater.cc:503] Master 127.4.40.62:45317 was elected leader, sending a full tablet report...
I20260811 06:56:01.641659 9401 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:32783
I20260811 06:56:01.641762 9631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:32783 every 8 connection(s)
I20260811 06:56:01.642932 9401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:01.652707 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9401
I20260811 06:56:01.652921 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:01.653396 4256 external_mini_cluster.cc:1428] Setting key 7e20b33fb6f7d3875968a632969eb2b4
Aug 11 06:56:01 dist-test-slave-7vjv krb5kdc[8688](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431361, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:01.671762 9634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:01.645369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:57821 (local address 127.4.40.62:45317)
0811 06:56:01.645809 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:01.645814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:01.646843 (+ 1029us) server_negotiation.cc:407] Connection header received
0811 06:56:01.649595 (+ 2752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:01.649599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:01.649662 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:01.649758 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:01.651227 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.651989 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.653649 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.654183 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.654309 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:01.665756 (+ 11447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:01.665760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:01.665763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:01.665807 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:01.669068 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:01.669767 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:01.669771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:01.669773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:01.669892 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:01.670285 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:01.670288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:01.670290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:01.670559 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:01.670667 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:01.671315 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 06:56:01.671482 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":208,"threads_started":1}
I20260811 06:56:01.672794 9637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:01.645971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45317 (local address 127.4.40.3:57821)
0811 06:56:01.646662 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:01.646699 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:01.649214 (+ 2515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:01.649957 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:01.649964 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:01.650456 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:01.650992 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:01.651019 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.652172 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.652175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:01.653455 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:01.653466 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:01.654426 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:01.654434 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:01.654764 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:01.663183 (+ 8419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:01.663213 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:01.665540 (+ 2327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:01.669198 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:01.669206 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:01.669219 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:01.669655 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:01.669984 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:01.669987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:01.669989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:01.670175 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:01.670666 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:01.670672 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:01.670991 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 06:56:01.671212 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 06:56:01.671601 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":279,"threads_started":1}
I20260811 06:56:01.674224 9632 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45317
I20260811 06:56:01.674520 9632 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:01.675222 9632 heartbeater.cc:511] Master 127.4.40.62:45317 requested a full tablet report, sending...
I20260811 06:56:01.676580 8767 ts_manager.cc:194] Registered new tserver with Master: 4a86ce20daf94055bae6b6ca33fb9da0 (127.4.40.3:32783)
I20260811 06:56:01.677366 8767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:57821
I20260811 06:56:01.681082 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:56:01.720180 9634 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:01.717403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.4.40.62:45317)
0811 06:56:01.717579 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:01.717583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:01.718084 (+ 501us) server_negotiation.cc:407] Connection header received
0811 06:56:01.718320 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:01.718324 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:01.718390 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:01.718522 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:01.719838 (+ 1316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34222: BlockingRecv error: recv got EOF from 127.0.0.1:34222 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:56:01.723323 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8884
W20260811 06:56:01.726217 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8884: Not authorized: Client connection negotiation failed: client connection to 127.4.40.1:43717: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:56:01.726390 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8884
I20260811 06:56:01.753858 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9158
W20260811 06:56:01.756796 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9158: Not authorized: Client connection negotiation failed: client connection to 127.4.40.2:40499: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:56:01.756966 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9158
I20260811 06:56:01.785073 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9401
W20260811 06:56:01.788239 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9401: Not authorized: Client connection negotiation failed: client connection to 127.4.40.3:32783: server requires authentication, but client does not have Kerberos credentials available
I20260811 06:56:01.788365 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9401
I20260811 06:56:01.813745 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8712
I20260811 06:56:01.815042 8816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:01.920239 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8712
2026-08-11T06:56:01Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4999 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:02 dist-test-slave-7vjv krb5kdc[9661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:02 dist-test-slave-7vjv krb5kdc[9661](info): set up 2 sockets
Aug 11 06:56:02 dist-test-slave-7vjv krb5kdc[9661](info): commencing operation
krb5kdc: starting...
W20260811 06:56:03.328231 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.000s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:03 dist-test-slave-7vjv krb5kdc[9661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:03.555682 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:38737
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:43373
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:38737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:03.883435 9689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:03.883714 9689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:03.883811 9689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:03.890898 9689 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:03.891000 9689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:03.891045 9689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:03.891104 9689 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:03.891153 9689 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:03.900939 9689 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43373
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:38737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:38737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:03.902132 9689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:03.903487 9689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:03.915450 9698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:03.915407 9697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:03.915668 9701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:03.916397 9689 server_base.cc:1034] running on GCE node
I20260811 06:56:03.917186 9689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:03.918495 9689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:03.919698 9689 hybrid_clock.cc:648] HybridClock initialized: now 1786431363919648 us; error 44 us; skew 500 ppm
I20260811 06:56:03.920040 9689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:03 dist-test-slave-7vjv krb5kdc[9661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431363, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:03.923288 9689 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:03.924782 9689 webserver.cc:460] Webserver started at http://127.4.40.62:34299/ using document root <none> and password file <none>
I20260811 06:56:03.925470 9689 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:03.925628 9689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:03.925990 9689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:03.928972 9689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "b794a67bc19c49d3a17d6b9073d72394"
format_stamp: "Formatted at 2026-08-11 06:56:03 on dist-test-slave-7vjv"
server_key: "c2e1e5d631e2e893f5f3abf9ac818fb5"
server_key_iv: "33715a3690e1da1e4f0d737f2fa94698"
server_key_version: "encryptionkey@0"
I20260811 06:56:03.929711 9689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "b794a67bc19c49d3a17d6b9073d72394"
format_stamp: "Formatted at 2026-08-11 06:56:03 on dist-test-slave-7vjv"
server_key: "c2e1e5d631e2e893f5f3abf9ac818fb5"
server_key_iv: "33715a3690e1da1e4f0d737f2fa94698"
server_key_version: "encryptionkey@0"
I20260811 06:56:03.934664 9689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:03.938082 9720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:03.939314 9689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 06:56:03.939447 9689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "b794a67bc19c49d3a17d6b9073d72394"
format_stamp: "Formatted at 2026-08-11 06:56:03 on dist-test-slave-7vjv"
server_key: "c2e1e5d631e2e893f5f3abf9ac818fb5"
server_key_iv: "33715a3690e1da1e4f0d737f2fa94698"
server_key_version: "encryptionkey@0"
I20260811 06:56:03.939628 9689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:03.969746 9689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:03.973601 9689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:03.973857 9689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:03.992805 9689 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:38737
I20260811 06:56:03.992851 9802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:38737 every 8 connection(s)
I20260811 06:56:03.994179 9689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:03.997264 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9689
I20260811 06:56:03.997488 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:03.998347 4256 external_mini_cluster.cc:1428] Setting key e8cbcffc1bc8c2b9dfd981d386aba59f
I20260811 06:56:03.998427 9803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:04.010733 9803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394: Bootstrap starting.
Aug 11 06:56:04 dist-test-slave-7vjv krb5kdc[9661](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431363, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:04.013803 9803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:04.015250 9803 log.cc:824] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:04.018551 9803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394: No bootstrap required, opened a new log
I20260811 06:56:04.021589 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:04.000343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35482 (local address 127.4.40.62:38737)
0811 06:56:04.001190 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:04.001201 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:04.001244 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:56:04.003153 (+ 1909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:04.003173 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:04.003547 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:04.003962 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:04.005544 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:04.006814 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:04.008144 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:04.008859 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:04.009100 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:04.013160 (+ 4060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:04.013167 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:04.013180 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:04.013214 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:04.016991 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:04.017678 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:04.017688 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:04.017698 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:04.017901 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:04.018265 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:04.018270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:04.018273 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:04.019076 (+ 803us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:04.019391 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:04.019755 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 06:56:04.020213 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":373,"threads_started":1}
I20260811 06:56:04.025473 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:56:04.027772 9803 raft_consensus.cc:348] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794a67bc19c49d3a17d6b9073d72394" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38737 } }
I20260811 06:56:04.028124 9803 raft_consensus.cc:374] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:04.028231 9803 raft_consensus.cc:729] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b794a67bc19c49d3a17d6b9073d72394, State: Initialized, Role: FOLLOWER
I20260811 06:56:04.028841 9803 consensus_queue.cc:260] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794a67bc19c49d3a17d6b9073d72394" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38737 } }
I20260811 06:56:04.029096 9803 raft_consensus.cc:388] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:04.029237 9803 raft_consensus.cc:482] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:04.029412 9803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:04.031469 9803 raft_consensus.cc:504] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794a67bc19c49d3a17d6b9073d72394" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38737 } }
I20260811 06:56:04.031981 9803 leader_election.cc:302] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b794a67bc19c49d3a17d6b9073d72394; no voters:
I20260811 06:56:04.032709 9803 leader_election.cc:288] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:04.032819 9808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:04.034055 9808 raft_consensus.cc:686] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [term 1 LEADER]: Becoming Leader. State: Replica: b794a67bc19c49d3a17d6b9073d72394, State: Running, Role: LEADER
I20260811 06:56:04.034696 9808 consensus_queue.cc:237] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794a67bc19c49d3a17d6b9073d72394" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38737 } }
I20260811 06:56:04.035135 9803 sys_catalog.cc:564] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:04.037822 9809 sys_catalog.cc:455] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b794a67bc19c49d3a17d6b9073d72394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794a67bc19c49d3a17d6b9073d72394" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38737 } } }
I20260811 06:56:04.037801 9812 sys_catalog.cc:455] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b794a67bc19c49d3a17d6b9073d72394. Latest consensus state: current_term: 1 leader_uuid: "b794a67bc19c49d3a17d6b9073d72394" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b794a67bc19c49d3a17d6b9073d72394" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38737 } } }
I20260811 06:56:04.038242 9809 sys_catalog.cc:458] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:04.038281 9812 sys_catalog.cc:458] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:04.051278 9827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:04.058298 9827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:04.068403 9827 catalog_manager.cc:1347] Generated new cluster ID: 214501562ffa4146a09b0cd785b01f2a
I20260811 06:56:04.068536 9827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:04.084398 9827 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:04.085852 9827 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:04.093909 9827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b794a67bc19c49d3a17d6b9073d72394: Generated new TSK 0
I20260811 06:56:04.094945 9827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:56:04.097168 9806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:04.085059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35492 (local address 127.4.40.62:38737)
0811 06:56:04.085232 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:04.085240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:04.088762 (+ 3522us) server_negotiation.cc:407] Connection header received
0811 06:56:04.088927 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:04.088932 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:04.089029 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:04.089150 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:04.090360 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:04.091527 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:04.093014 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:04.093712 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:04.093861 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:04.095251 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:04.095264 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:04.095268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:04.096376 (+ 1108us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:04.096431 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:04.096464 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:04.096577 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:04.096720 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:56:04.096816 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:56:04.104832 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9689
I20260811 06:56:04.106168 9798 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:04.194278 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9689
2026-08-11T06:56:04Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2266 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:04 dist-test-slave-7vjv krb5kdc[9870](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:04 dist-test-slave-7vjv krb5kdc[9870](info): set up 2 sockets
Aug 11 06:56:04 dist-test-slave-7vjv krb5kdc[9870](info): commencing operation
krb5kdc: starting...
W20260811 06:56:05.561919 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:05 dist-test-slave-7vjv krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:05.791500 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43347
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:43347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:06.118074 9902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:06.118351 9902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:06.118525 9902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:06.125514 9902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:06.125638 9902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:06.125739 9902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:06.125865 9902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:06.125960 9902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:06.136072 9902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:43347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:06.137353 9902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:06.138651 9902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:06.150321 9911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:06.150383 9912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:06.150429 9916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:06.151566 9902 server_base.cc:1034] running on GCE node
I20260811 06:56:06.152393 9902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:06.153688 9902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:06.154886 9902 hybrid_clock.cc:648] HybridClock initialized: now 1786431366154823 us; error 47 us; skew 500 ppm
I20260811 06:56:06.155221 9902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:06 dist-test-slave-7vjv krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431366, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:06.158390 9902 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:06.159909 9902 webserver.cc:460] Webserver started at http://127.4.40.62:33737/ using document root <none> and password file <none>
I20260811 06:56:06.160598 9902 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:06.160701 9902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:06.160964 9902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:06.163733 9902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "f7b06742e2604c999c1ebd724b2b0541"
format_stamp: "Formatted at 2026-08-11 06:56:06 on dist-test-slave-7vjv"
server_key: "988db605cad761094df15f9b455debb8"
server_key_iv: "629ed2307741fae0e96063d783609795"
server_key_version: "encryptionkey@0"
I20260811 06:56:06.164515 9902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "f7b06742e2604c999c1ebd724b2b0541"
format_stamp: "Formatted at 2026-08-11 06:56:06 on dist-test-slave-7vjv"
server_key: "988db605cad761094df15f9b455debb8"
server_key_iv: "629ed2307741fae0e96063d783609795"
server_key_version: "encryptionkey@0"
I20260811 06:56:06.169543 9902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:56:06.172994 9930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:06.174288 9902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:56:06.174474 9902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "f7b06742e2604c999c1ebd724b2b0541"
format_stamp: "Formatted at 2026-08-11 06:56:06 on dist-test-slave-7vjv"
server_key: "988db605cad761094df15f9b455debb8"
server_key_iv: "629ed2307741fae0e96063d783609795"
server_key_version: "encryptionkey@0"
I20260811 06:56:06.174667 9902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:06.191704 9902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:06.195674 9902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:06.195989 9902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:06.215169 9902 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:43347
I20260811 06:56:06.215206 10008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:43347 every 8 connection(s)
I20260811 06:56:06.216696 9902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:06.220382 10013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:06.220953 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9902
I20260811 06:56:06.221186 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:06.222062 4256 external_mini_cluster.cc:1428] Setting key b2a79c2fe0fd4b2367db75b16f77c192
I20260811 06:56:06.234817 10013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541: Bootstrap starting.
Aug 11 06:56:06 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:06.238489 10013 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:06.239768 10013 log.cc:824] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:06.242749 10013 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541: No bootstrap required, opened a new log
I20260811 06:56:06.245626 10017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:06.224328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.4.40.62:43347)
0811 06:56:06.225126 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:06.225137 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:06.225182 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:56:06.226967 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:06.226988 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:06.227364 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:06.227689 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:06.229506 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:06.230758 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:06.232583 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:06.233231 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:06.233365 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:06.238134 (+ 4769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:06.238141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:06.238155 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:06.238189 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:06.241486 (+ 3297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:06.242184 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:06.242193 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:06.242201 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:06.242381 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:06.242779 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:06.242784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:06.242788 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:06.243351 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:06.243702 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:06.244190 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 06:56:06.244624 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":372,"threads_started":1}
I20260811 06:56:06.250998 10013 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b06742e2604c999c1ebd724b2b0541" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43347 } }
I20260811 06:56:06.251348 10013 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:06.251467 10013 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b06742e2604c999c1ebd724b2b0541, State: Initialized, Role: FOLLOWER
I20260811 06:56:06.251997 10013 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b06742e2604c999c1ebd724b2b0541" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43347 } }
I20260811 06:56:06.252305 10013 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:06.252465 10013 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:06.252645 10013 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:06.254855 10013 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b06742e2604c999c1ebd724b2b0541" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43347 } }
I20260811 06:56:06.255339 10013 leader_election.cc:302] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7b06742e2604c999c1ebd724b2b0541; no voters:
I20260811 06:56:06.255945 10013 leader_election.cc:288] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:06.256052 10020 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:06.257328 10020 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [term 1 LEADER]: Becoming Leader. State: Replica: f7b06742e2604c999c1ebd724b2b0541, State: Running, Role: LEADER
I20260811 06:56:06.257833 10020 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b06742e2604c999c1ebd724b2b0541" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43347 } }
I20260811 06:56:06.258150 10013 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:06.260746 10025 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7b06742e2604c999c1ebd724b2b0541. Latest consensus state: current_term: 1 leader_uuid: "f7b06742e2604c999c1ebd724b2b0541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b06742e2604c999c1ebd724b2b0541" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43347 } } }
I20260811 06:56:06.261437 10025 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:06.260986 10024 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7b06742e2604c999c1ebd724b2b0541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7b06742e2604c999c1ebd724b2b0541" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43347 } } }
I20260811 06:56:06.261713 10024 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:06.266947 10032 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:06.271859 10032 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:06.280069 10032 catalog_manager.cc:1347] Generated new cluster ID: bd437a30dfe049ebb9d040fb072be14b
I20260811 06:56:06.280206 10032 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:06.302665 10032 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:06.304040 10032 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:06.312920 10032 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7b06742e2604c999c1ebd724b2b0541: Generated new TSK 0
I20260811 06:56:06.313789 10032 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:56:06.373906 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43347
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:06.718729 10051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:06.719062 10051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:06.719235 10051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:06.726501 10051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:06.726648 10051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:06.726833 10051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:06.736068 10051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:06.737377 10051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:06.738785 10051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:06.752780 10061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:06.752887 10066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:06.752846 10064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:06.753868 10051 server_base.cc:1034] running on GCE node
I20260811 06:56:06.754469 10051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:06.755327 10051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:06.756633 10051 hybrid_clock.cc:648] HybridClock initialized: now 1786431366756577 us; error 30 us; skew 500 ppm
I20260811 06:56:06.756975 10051 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:06 dist-test-slave-7vjv krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431366, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:06.764406 10051 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:56:06.766074 10051 webserver.cc:460] Webserver started at http://127.4.40.1:37299/ using document root <none> and password file <none>
I20260811 06:56:06.766819 10051 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:06.766961 10051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:06.767298 10051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:06.770406 10051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "eb5bfb69514f44568b06990864c92555"
format_stamp: "Formatted at 2026-08-11 06:56:06 on dist-test-slave-7vjv"
server_key: "b7a35c85080f73a7b0c4590daa0277b3"
server_key_iv: "d133280f9c6199bda566da8d92b0abad"
server_key_version: "encryptionkey@0"
I20260811 06:56:06.771214 10051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "eb5bfb69514f44568b06990864c92555"
format_stamp: "Formatted at 2026-08-11 06:56:06 on dist-test-slave-7vjv"
server_key: "b7a35c85080f73a7b0c4590daa0277b3"
server_key_iv: "d133280f9c6199bda566da8d92b0abad"
server_key_version: "encryptionkey@0"
I20260811 06:56:06.777036 10051 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:56:06.780616 10079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:06.781930 10051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:56:06.782122 10051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "eb5bfb69514f44568b06990864c92555"
format_stamp: "Formatted at 2026-08-11 06:56:06 on dist-test-slave-7vjv"
server_key: "b7a35c85080f73a7b0c4590daa0277b3"
server_key_iv: "d133280f9c6199bda566da8d92b0abad"
server_key_version: "encryptionkey@0"
I20260811 06:56:06.782327 10051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:06.812043 10051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:06.817132 10051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:06.817471 10051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:06.818506 10051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:06.819979 10051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:06.820094 10051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:06.820281 10051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:06.820370 10051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:06.854039 10051 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:45343
I20260811 06:56:06.854151 10274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:45343 every 8 connection(s)
I20260811 06:56:06.855444 10051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:06.857800 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10051
I20260811 06:56:06.858039 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:06.858930 4256 external_mini_cluster.cc:1428] Setting key 9d8976af2225598d9aee732780285d99
Aug 11 06:56:06 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431366, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:06.878728 10277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:06.858157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:51067 (local address 127.4.40.62:43347)
0811 06:56:06.858714 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:06.858719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:06.859952 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 06:56:06.862183 (+ 2231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:06.862187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:06.862247 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:06.862338 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:06.864496 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:06.865549 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:06.868202 (+ 2653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:06.868850 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:06.868982 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:06.872862 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:06.872867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:06.872871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:06.872903 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:06.875525 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:06.876312 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:06.876316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:06.876318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:06.876438 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:06.876966 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:06.876970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:06.876972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:06.877227 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:06.877385 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:06.878165 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 06:56:06.878381 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":322,"threads_started":1}
I20260811 06:56:06.879807 10279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:06.858804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43347 (local address 127.4.40.1:51067)
0811 06:56:06.859760 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:06.859809 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:06.861894 (+ 2085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:06.862630 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:06.862643 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:06.863433 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:06.864279 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:06.864306 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:06.865739 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:06.865747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:06.868002 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:06.868021 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:06.869087 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:06.869096 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:06.869419 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:06.870214 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:06.870248 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:06.872633 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:06.875670 (+ 3037us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:06.875678 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:06.875692 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:06.876174 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:06.876573 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:06.876595 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:06.876599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:06.876822 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:06.877358 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:06.877366 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:06.877764 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 06:56:06.878019 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 06:56:06.878501 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":395,"threads_started":1}
I20260811 06:56:06.881384 10275 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43347
I20260811 06:56:06.881711 10275 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:06.882423 10275 heartbeater.cc:511] Master 127.4.40.62:43347 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
I20260811 06:56:06.884758 9957 ts_manager.cc:194] Registered new tserver with Master: eb5bfb69514f44568b06990864c92555 (127.4.40.1:45343)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
I20260811 06:56:06.886463 9957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:51067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:56:06.915186 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43347
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:07.263595 10283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:07.263885 10283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:07.263995 10283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:07.271461 10283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:07.271572 10283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:07.271719 10283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:07.281980 10283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:07.283221 10283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:07.284616 10283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:07.298276 10291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:07.298322 10292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:07.298159 10296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:07.299185 10283 server_base.cc:1034] running on GCE node
I20260811 06:56:07.300289 10283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:07.301184 10283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:07.302394 10283 hybrid_clock.cc:648] HybridClock initialized: now 1786431367302339 us; error 48 us; skew 500 ppm
I20260811 06:56:07.302711 10283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:07.306165 10283 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:56:07.307718 10283 webserver.cc:460] Webserver started at http://127.4.40.2:33715/ using document root <none> and password file <none>
I20260811 06:56:07.308557 10283 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:07.308753 10283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:07.309109 10283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:07.312238 10283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "420dcd513aed48089b41636abc42c868"
format_stamp: "Formatted at 2026-08-11 06:56:07 on dist-test-slave-7vjv"
server_key: "0b1e4cd67ff5285b6e172d7f97e64859"
server_key_iv: "4f39bd08e8c355b2bfbbfb3602f0042b"
server_key_version: "encryptionkey@0"
I20260811 06:56:07.313225 10283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "420dcd513aed48089b41636abc42c868"
format_stamp: "Formatted at 2026-08-11 06:56:07 on dist-test-slave-7vjv"
server_key: "0b1e4cd67ff5285b6e172d7f97e64859"
server_key_iv: "4f39bd08e8c355b2bfbbfb3602f0042b"
server_key_version: "encryptionkey@0"
I20260811 06:56:07.318614 10283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260811 06:56:07.322122 10306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:07.323489 10283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:56:07.323678 10283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "420dcd513aed48089b41636abc42c868"
format_stamp: "Formatted at 2026-08-11 06:56:07 on dist-test-slave-7vjv"
server_key: "0b1e4cd67ff5285b6e172d7f97e64859"
server_key_iv: "4f39bd08e8c355b2bfbbfb3602f0042b"
server_key_version: "encryptionkey@0"
I20260811 06:56:07.323884 10283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:07.349174 10283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:07.353670 10283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:07.353983 10283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:07.354981 10283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:07.356387 10283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:07.356464 10283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:07.356643 10283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:07.356698 10283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:07.391446 10283 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:36765
I20260811 06:56:07.391515 10512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:36765 every 8 connection(s)
I20260811 06:56:07.392786 10283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:07.398470 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10283
I20260811 06:56:07.398667 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:07.399296 4256 external_mini_cluster.cc:1428] Setting key 213466fc55df0271443d0755bdcc6273
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:07.413131 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:07.395297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:44773 (local address 127.4.40.62:43347)
0811 06:56:07.395713 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:07.395718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:07.396717 (+ 999us) server_negotiation.cc:407] Connection header received
0811 06:56:07.398765 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:07.398771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:07.398851 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:07.398946 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:07.400833 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.401719 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.403296 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.403717 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.403859 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:07.407410 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:07.407415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:07.407418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:07.407452 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:07.410045 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:07.410774 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:07.410778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:07.410781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:07.410903 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:07.411408 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:07.411412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:07.411414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:07.411675 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:07.411788 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:07.412694 (+ 906us) server_negotiation.cc:299] Negotiation successful
0811 06:56:07.412863 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":201,"threads_started":1}
I20260811 06:56:07.414706 10516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:07.395809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43347 (local address 127.4.40.2:44773)
0811 06:56:07.396555 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:07.396602 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:07.398425 (+ 1823us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:07.399217 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:07.399228 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:07.400002 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:07.400652 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:07.400671 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.401886 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.401890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:07.403140 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:07.403153 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.403984 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.403992 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:07.404288 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:07.404999 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:07.405022 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:07.407185 (+ 2163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:07.410186 (+ 3001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:07.410200 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:07.410215 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:07.410638 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:07.411020 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:07.411027 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:07.411030 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:07.411261 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:07.411821 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:07.411832 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:07.412292 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 06:56:07.412597 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 06:56:07.413155 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":320,"threads_started":1}
I20260811 06:56:07.416234 10513 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43347
I20260811 06:56:07.416555 10513 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:07.417296 10513 heartbeater.cc:511] Master 127.4.40.62:43347 requested a full tablet report, sending...
I20260811 06:56:07.418780 9957 ts_manager.cc:194] Registered new tserver with Master: 420dcd513aed48089b41636abc42c868 (127.4.40.2:36765)
I20260811 06:56:07.419567 9957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:44773
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:56:07.454749 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43347
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:07.785631 10520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:07.785935 10520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:07.786123 10520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:07.793048 10520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:07.793195 10520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:07.793386 10520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:07.802438 10520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43347
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:07.803635 10520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:07.804977 10520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:07.818312 10534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:07.818548 10536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:07.818845 10529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:07.818900 10520 server_base.cc:1034] running on GCE node
I20260811 06:56:07.819795 10520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:07.820709 10520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:07.821913 10520 hybrid_clock.cc:648] HybridClock initialized: now 1786431367821867 us; error 37 us; skew 500 ppm
I20260811 06:56:07.822247 10520 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:07.825409 10520 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:56:07.826915 10520 webserver.cc:460] Webserver started at http://127.4.40.3:42201/ using document root <none> and password file <none>
I20260811 06:56:07.827620 10520 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:07.827766 10520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:07.828173 10520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:07.830914 10520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "2ed8fd8b926f41c685f618e8d879978f"
format_stamp: "Formatted at 2026-08-11 06:56:07 on dist-test-slave-7vjv"
server_key: "abe29c9ddcb26a81dd99e28909da47c0"
server_key_iv: "761180f44eee03d0aaeccdf3f3a8b463"
server_key_version: "encryptionkey@0"
I20260811 06:56:07.831705 10520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "2ed8fd8b926f41c685f618e8d879978f"
format_stamp: "Formatted at 2026-08-11 06:56:07 on dist-test-slave-7vjv"
server_key: "abe29c9ddcb26a81dd99e28909da47c0"
server_key_iv: "761180f44eee03d0aaeccdf3f3a8b463"
server_key_version: "encryptionkey@0"
I20260811 06:56:07.837287 10520 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 06:56:07.840893 10551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:07.842106 10520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:56:07.842289 10520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "2ed8fd8b926f41c685f618e8d879978f"
format_stamp: "Formatted at 2026-08-11 06:56:07 on dist-test-slave-7vjv"
server_key: "abe29c9ddcb26a81dd99e28909da47c0"
server_key_iv: "761180f44eee03d0aaeccdf3f3a8b463"
server_key_version: "encryptionkey@0"
I20260811 06:56:07.842506 10520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:07.862102 10520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:07.865939 10520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:07.866251 10520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:07.867233 10520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:07.868606 10520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:07.868712 10520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:07.868863 10520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:07.868939 10520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:07.889868 10275 heartbeater.cc:503] Master 127.4.40.62:43347 was elected leader, sending a full tablet report...
I20260811 06:56:07.903501 10520 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:38497
I20260811 06:56:07.903685 10767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:38497 every 8 connection(s)
I20260811 06:56:07.904904 10520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:07.905279 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10520
I20260811 06:56:07.905491 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:07.906173 4256 external_mini_cluster.cc:1428] Setting key 81c8b6b7f69840abf7b3c8a323f06dea
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:07.925949 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:07.907800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:59761 (local address 127.4.40.62:43347)
0811 06:56:07.907987 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:07.907991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:07.909358 (+ 1367us) server_negotiation.cc:407] Connection header received
0811 06:56:07.911331 (+ 1973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:07.911337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:07.911438 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:07.911560 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:07.913139 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.913885 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.915445 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.915872 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.916031 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:07.919692 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:07.919697 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:07.919700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:07.919734 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:07.922840 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:07.923576 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:07.923588 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:07.923591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:07.923714 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:07.924184 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:07.924189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:07.924192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:07.924502 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:07.924673 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:07.925363 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 06:56:07.925608 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:56:07.926997 10772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:07.908389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43347 (local address 127.4.40.3:59761)
0811 06:56:07.909189 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:07.909235 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:07.911069 (+ 1834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:07.911804 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:07.911813 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:07.912412 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:07.912953 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:07.912970 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.914068 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.914072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:07.915290 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:07.915302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.916180 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.916189 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:07.916423 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:07.917132 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:07.917160 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:07.919484 (+ 2324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:07.922984 (+ 3500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:07.922993 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:07.923008 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:07.923446 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:07.923827 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:07.923831 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:07.923835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:07.924027 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:07.924644 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:07.924652 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:07.924988 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 06:56:07.925255 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 06:56:07.925767 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":341,"threads_started":1}
I20260811 06:56:07.928462 10769 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43347
I20260811 06:56:07.928783 10769 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:07.929553 10769 heartbeater.cc:511] Master 127.4.40.62:43347 requested a full tablet report, sending...
I20260811 06:56:07.930936 9957 ts_manager.cc:194] Registered new tserver with Master: 2ed8fd8b926f41c685f618e8d879978f (127.4.40.3:38497)
I20260811 06:56:07.931715 9957 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:59761
I20260811 06:56:07.934177 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:07.990042 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:07.975869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.4.40.62:43347)
0811 06:56:07.976153 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:07.976162 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:07.976560 (+ 398us) server_negotiation.cc:407] Connection header received
0811 06:56:07.976726 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:07.976730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:07.976798 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:07.976883 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:07.978292 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.979089 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.980460 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.980864 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.981004 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:07.984953 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:07.984958 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:07.984961 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:07.984996 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:07.987688 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:07.988331 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:07.988335 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:07.988337 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:07.988446 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:07.988778 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:07.988781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:07.988783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:07.989097 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:07.989287 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:07.989515 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 06:56:07.989677 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 06:56:07.990664 9957 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41972
I20260811 06:56:07.992625 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10051
Aug 11 06:56:07 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:08.008890 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:07.993652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40248 (local address 127.4.40.1:45343)
0811 06:56:07.994325 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:07.994333 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:07.994356 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:56:07.994412 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:07.994418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:07.994625 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:07.994806 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:07.996006 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.997086 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.998267 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:07.998869 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:07.999008 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:08.002944 (+ 3936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:08.002955 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:08.002971 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:08.003007 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:08.006108 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:08.006757 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:08.006772 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:08.006775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:08.006905 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:08.007278 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:08.007283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:08.007287 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:08.007627 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:08.007934 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:08.008286 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 06:56:08.008565 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":314,"threads_started":1}
W20260811 06:56:08.010123 10264 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40248
W20260811 06:56:08.011204 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10051: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:56:08.011405 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10051
I20260811 06:56:08.038570 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10283
Aug 11 06:56:08 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:56:08.054486 10790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:08.039526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.4.40.2:36765)
0811 06:56:08.040198 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:08.040205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:08.040233 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:56:08.040295 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:08.040302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:08.040540 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:08.040666 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:08.041831 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:08.042831 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:08.043977 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:08.044614 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:08.044720 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:08.048715 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:08.048726 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:08.048738 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:08.048770 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:08.051655 (+ 2885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:08.052394 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:08.052402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:08.052404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:08.052538 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:08.052909 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:08.052914 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:08.052917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:08.053274 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:08.053623 (+ 349us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:08.053862 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 06:56:08.054138 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":332,"threads_started":1}
W20260811 06:56:08.055727 10506 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41102
W20260811 06:56:08.056905 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10283: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:56:08.057070 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10283
I20260811 06:56:08.087178 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10520
Aug 11 06:56:08 dist-test-slave-7vjv krb5kdc[9870](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431367, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:56:08.103439 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:08.088294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55062 (local address 127.4.40.3:38497)
0811 06:56:08.088998 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:08.089005 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:08.089033 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:56:08.089109 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:08.089115 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:08.089312 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:08.089449 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:08.090697 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:08.091669 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:08.092984 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:08.093507 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:08.093603 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:08.098277 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:08.098287 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:08.098299 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:08.098334 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:08.101072 (+ 2738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:08.101670 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:08.101676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:08.101678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:08.101811 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:08.102108 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:08.102111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:08.102113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:08.102432 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:08.102667 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:08.102861 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 06:56:08.103045 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":378,"threads_started":1}
W20260811 06:56:08.104782 10757 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:55062
W20260811 06:56:08.105921 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10520: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 06:56:08.106108 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10520
I20260811 06:56:08.128901 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9902
I20260811 06:56:08.130030 10002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:08.233915 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9902
2026-08-11T06:56:08Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4044 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:08 dist-test-slave-7vjv krb5kdc[10802](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:08 dist-test-slave-7vjv krb5kdc[10802](info): set up 2 sockets
Aug 11 06:56:08 dist-test-slave-7vjv krb5kdc[10802](info): commencing operation
krb5kdc: starting...
W20260811 06:56:09.604238 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:09 dist-test-slave-7vjv krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:09.833611 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:44223
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:44223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:10.166930 10821 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:10.167214 10821 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:10.167389 10821 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:10.174729 10821 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:10.174863 10821 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:10.174952 10821 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:10.175056 10821 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:10.175141 10821 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:10.185093 10821 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:44223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:44223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:10.186385 10821 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:10.187700 10821 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:10.199594 10832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:10.199638 10828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:10.199672 10829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:10.200788 10821 server_base.cc:1034] running on GCE node
I20260811 06:56:10.201589 10821 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:10.202941 10821 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:10.204224 10821 hybrid_clock.cc:648] HybridClock initialized: now 1786431370204167 us; error 41 us; skew 500 ppm
I20260811 06:56:10.204588 10821 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:10 dist-test-slave-7vjv krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:10.207687 10821 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:10.209197 10821 webserver.cc:460] Webserver started at http://127.4.40.62:44847/ using document root <none> and password file <none>
I20260811 06:56:10.209962 10821 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:10.210134 10821 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:10.210433 10821 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:10.213598 10821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "5110a910db054364b9c465d73a1dd943"
format_stamp: "Formatted at 2026-08-11 06:56:10 on dist-test-slave-7vjv"
server_key: "743dca9d39fd36b539ed3b6c74491bf8"
server_key_iv: "557de252b3a9f2bcc1b55559641882c4"
server_key_version: "encryptionkey@0"
I20260811 06:56:10.214361 10821 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "5110a910db054364b9c465d73a1dd943"
format_stamp: "Formatted at 2026-08-11 06:56:10 on dist-test-slave-7vjv"
server_key: "743dca9d39fd36b539ed3b6c74491bf8"
server_key_iv: "557de252b3a9f2bcc1b55559641882c4"
server_key_version: "encryptionkey@0"
I20260811 06:56:10.219990 10821 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:56:10.223531 10847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:10.224888 10821 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:10.225099 10821 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "5110a910db054364b9c465d73a1dd943"
format_stamp: "Formatted at 2026-08-11 06:56:10 on dist-test-slave-7vjv"
server_key: "743dca9d39fd36b539ed3b6c74491bf8"
server_key_iv: "557de252b3a9f2bcc1b55559641882c4"
server_key_version: "encryptionkey@0"
I20260811 06:56:10.225314 10821 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:10.240294 10821 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:10.244046 10821 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:10.244390 10821 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:10.263471 10821 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:44223
I20260811 06:56:10.263504 10947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:44223 every 8 connection(s)
I20260811 06:56:10.264940 10821 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:10.268698 10948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:10.274602 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10821
I20260811 06:56:10.274873 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:10.275701 4256 external_mini_cluster.cc:1428] Setting key 5e17e0b713d71c9f13c711465e6331d2
I20260811 06:56:10.280215 10948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943: Bootstrap starting.
I20260811 06:56:10.283957 10948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:10.285181 10948 log.cc:824] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:10.288256 10948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943: No bootstrap required, opened a new log
Aug 11 06:56:10 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431369, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:10.296211 10948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5110a910db054364b9c465d73a1dd943" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44223 } }
I20260811 06:56:10.296555 10948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:10.296689 10948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5110a910db054364b9c465d73a1dd943, State: Initialized, Role: FOLLOWER
I20260811 06:56:10.297320 10948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5110a910db054364b9c465d73a1dd943" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44223 } }
I20260811 06:56:10.297626 10948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:10.297789 10948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:10.297996 10948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:10.299242 10951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:10.277865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37204 (local address 127.4.40.62:44223)
0811 06:56:10.278695 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:10.278707 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:10.278746 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:56:10.280601 (+ 1855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:10.280622 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:10.280971 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:10.281301 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:10.283145 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:10.284517 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:10.286230 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:10.286924 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:10.287056 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:10.291708 (+ 4652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:10.291715 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:10.291729 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:10.291765 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:10.294959 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:10.295652 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:10.295659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:10.295665 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:10.295835 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:10.296301 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:10.296306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:10.296313 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:10.297004 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:10.297288 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:10.297809 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 06:56:10.298244 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":402,"threads_started":1}
I20260811 06:56:10.300531 10948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5110a910db054364b9c465d73a1dd943" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44223 } }
I20260811 06:56:10.301011 10948 leader_election.cc:302] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5110a910db054364b9c465d73a1dd943; no voters:
I20260811 06:56:10.301620 10948 leader_election.cc:288] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:10.301772 10956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:10.302999 10956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [term 1 LEADER]: Becoming Leader. State: Replica: 5110a910db054364b9c465d73a1dd943, State: Running, Role: LEADER
I20260811 06:56:10.303509 10956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5110a910db054364b9c465d73a1dd943" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44223 } }
I20260811 06:56:10.303718 10948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:10.308369 10958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5110a910db054364b9c465d73a1dd943. Latest consensus state: current_term: 1 leader_uuid: "5110a910db054364b9c465d73a1dd943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5110a910db054364b9c465d73a1dd943" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44223 } } }
I20260811 06:56:10.308728 10958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:10.309214 10957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5110a910db054364b9c465d73a1dd943" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5110a910db054364b9c465d73a1dd943" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44223 } } }
I20260811 06:56:10.309527 10957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:10.310745 10972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:10.315353 10972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:10.323552 10972 catalog_manager.cc:1347] Generated new cluster ID: d34364362f154821947093e6a0a172bf
I20260811 06:56:10.323657 10972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:10.343451 10972 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:10.344834 10972 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:10.354516 10972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5110a910db054364b9c465d73a1dd943: Generated new TSK 0
I20260811 06:56:10.355260 10972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:56:10.411803 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44223
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:10.752396 10988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:10.752794 10988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:10.752993 10988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:10.760174 10988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:10.760311 10988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:10.760504 10988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:10.770357 10988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:10.771703 10988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:10.773206 10988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:10.787209 10998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:10.787257 10999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:10.787284 11004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:10.788210 10988 server_base.cc:1034] running on GCE node
I20260811 06:56:10.788786 10988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:10.789669 10988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:10.790870 10988 hybrid_clock.cc:648] HybridClock initialized: now 1786431370790825 us; error 40 us; skew 500 ppm
I20260811 06:56:10.791152 10988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:10 dist-test-slave-7vjv krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:10.794361 10988 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:56:10.795842 10988 webserver.cc:460] Webserver started at http://127.4.40.1:34485/ using document root <none> and password file <none>
I20260811 06:56:10.796576 10988 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:10.796713 10988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:10.797039 10988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:10.800287 10988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "bc2ac04f97974286b66b687a3e00446d"
format_stamp: "Formatted at 2026-08-11 06:56:10 on dist-test-slave-7vjv"
server_key: "4cde042cecd31f3f2a4ebc9987ca184a"
server_key_iv: "c6f569f7032e5cd4ed0239940de20f87"
server_key_version: "encryptionkey@0"
I20260811 06:56:10.801110 10988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "bc2ac04f97974286b66b687a3e00446d"
format_stamp: "Formatted at 2026-08-11 06:56:10 on dist-test-slave-7vjv"
server_key: "4cde042cecd31f3f2a4ebc9987ca184a"
server_key_iv: "c6f569f7032e5cd4ed0239940de20f87"
server_key_version: "encryptionkey@0"
I20260811 06:56:10.806432 10988 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:10.810114 11018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:10.811530 10988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:56:10.811704 10988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "bc2ac04f97974286b66b687a3e00446d"
format_stamp: "Formatted at 2026-08-11 06:56:10 on dist-test-slave-7vjv"
server_key: "4cde042cecd31f3f2a4ebc9987ca184a"
server_key_iv: "c6f569f7032e5cd4ed0239940de20f87"
server_key_version: "encryptionkey@0"
I20260811 06:56:10.811911 10988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:10.844604 10988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:10.848482 10988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:10.848767 10988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:10.849766 10988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:10.851105 10988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:10.851176 10988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:10.851336 10988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:10.851403 10988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:10.885190 10988 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:35991
I20260811 06:56:10.885211 11203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:35991 every 8 connection(s)
I20260811 06:56:10.886462 10988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:10.895150 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10988
I20260811 06:56:10.895382 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:10.896066 4256 external_mini_cluster.cc:1428] Setting key 66f42e06c6f93515006496b3ade03260
Aug 11 06:56:10 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431370, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:10.906840 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:10.888977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:42265 (local address 127.4.40.62:44223)
0811 06:56:10.889400 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:10.889405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:10.890473 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 06:56:10.892301 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:10.892307 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:10.892395 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:10.892525 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:10.894019 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:10.894748 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:10.896520 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:10.897013 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:10.897135 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:10.900946 (+ 3811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:10.900951 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:10.900954 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:10.900985 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:10.903604 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:10.904397 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:10.904401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:10.904404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:10.904558 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:10.904998 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:10.905003 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:10.905007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:10.905328 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:10.905488 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:10.906207 (+ 719us) server_negotiation.cc:299] Negotiation successful
0811 06:56:10.906427 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":205,"threads_started":1}
I20260811 06:56:10.907819 11210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:10.889539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44223 (local address 127.4.40.1:42265)
0811 06:56:10.890256 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:10.890312 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:10.892005 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:10.892763 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:10.892771 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:10.893328 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:10.893847 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:10.893863 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:10.894926 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:10.894930 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:10.896376 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:10.896389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:10.897266 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:10.897274 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:10.897566 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:10.898309 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:10.898330 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:10.900724 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:10.903733 (+ 3009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:10.903742 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:10.903756 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:10.904273 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:10.904670 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:10.904675 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:10.904679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:10.904878 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:10.905453 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:10.905461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:10.905779 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 06:56:10.906075 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 06:56:10.906593 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":285,"threads_started":1}
I20260811 06:56:10.909397 11204 heartbeater.cc:347] Connected to a master server at 127.4.40.62:44223
I20260811 06:56:10.909729 11204 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:10.910610 11204 heartbeater.cc:511] Master 127.4.40.62:44223 requested a full tablet report, sending...
I20260811 06:56:10.912963 10880 ts_manager.cc:194] Registered new tserver with Master: bc2ac04f97974286b66b687a3e00446d (127.4.40.1:35991)
I20260811 06:56:10.915166 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:42265
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:56:10.951130 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44223
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:11.281328 11215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:11.281579 11215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:11.281667 11215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:11.288955 11215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:11.289072 11215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:11.289188 11215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:11.298611 11215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:11.299767 11215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:11.301111 11215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:11.314734 11228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:11.314895 11231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:11.314723 11229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:11.316020 11215 server_base.cc:1034] running on GCE node
I20260811 06:56:11.316733 11215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:11.317698 11215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:11.318929 11215 hybrid_clock.cc:648] HybridClock initialized: now 1786431371318870 us; error 50 us; skew 500 ppm
I20260811 06:56:11.319248 11215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:11 dist-test-slave-7vjv krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:11.322408 11215 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:56:11.323944 11215 webserver.cc:460] Webserver started at http://127.4.40.2:46361/ using document root <none> and password file <none>
I20260811 06:56:11.324697 11215 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:11.324844 11215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:11.325171 11215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:11.328019 11215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "f9fa38e15a544d3da90e02176dc74879"
format_stamp: "Formatted at 2026-08-11 06:56:11 on dist-test-slave-7vjv"
server_key: "7af470cd95e80b34748275dbf4afa958"
server_key_iv: "a2d3d5332666d3d0b14a2f4ae8bb818c"
server_key_version: "encryptionkey@0"
I20260811 06:56:11.328902 11215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "f9fa38e15a544d3da90e02176dc74879"
format_stamp: "Formatted at 2026-08-11 06:56:11 on dist-test-slave-7vjv"
server_key: "7af470cd95e80b34748275dbf4afa958"
server_key_iv: "a2d3d5332666d3d0b14a2f4ae8bb818c"
server_key_version: "encryptionkey@0"
I20260811 06:56:11.334272 11215 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:11.337653 11242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:11.339004 11215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:56:11.339185 11215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "f9fa38e15a544d3da90e02176dc74879"
format_stamp: "Formatted at 2026-08-11 06:56:11 on dist-test-slave-7vjv"
server_key: "7af470cd95e80b34748275dbf4afa958"
server_key_iv: "a2d3d5332666d3d0b14a2f4ae8bb818c"
server_key_version: "encryptionkey@0"
I20260811 06:56:11.339389 11215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:11.351136 11215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:11.354889 11215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:11.355186 11215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:11.356366 11215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:11.357798 11215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:11.357903 11215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:11.358034 11215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:11.358112 11215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:11.391986 11215 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:34209
I20260811 06:56:11.392040 11468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:34209 every 8 connection(s)
I20260811 06:56:11.393302 11215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:11.403249 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11215
I20260811 06:56:11.403492 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:11.404060 4256 external_mini_cluster.cc:1428] Setting key 50de5ae7bfc2211e5ea85ff1de858372
Aug 11 06:56:11 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:11.415120 11206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:11.395764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:57563 (local address 127.4.40.62:44223)
0811 06:56:11.396012 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:11.396019 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:11.397313 (+ 1294us) server_negotiation.cc:407] Connection header received
0811 06:56:11.399385 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:11.399391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:11.399493 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:11.399613 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:11.401158 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.401875 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.403482 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.404204 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.404349 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:11.407944 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:11.407953 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:11.407962 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:11.408008 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:11.411717 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:11.412574 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:11.412582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:11.412587 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:11.412753 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:11.413204 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:11.413212 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:11.413217 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:11.413592 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:11.413747 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:11.414371 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 06:56:11.414648 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 06:56:11.415786 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:11.396290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44223 (local address 127.4.40.2:57563)
0811 06:56:11.397052 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:11.397142 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:11.399127 (+ 1985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:11.399844 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:11.399851 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:11.400441 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:11.400995 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:11.401011 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.402047 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.402051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:11.403250 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:11.403266 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.404500 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.404523 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:11.404741 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:11.405416 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:11.405438 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:11.407686 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:11.411850 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:11.411857 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:11.411868 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:11.412459 (+ 591us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:11.412869 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:11.412872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:11.412874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:11.413090 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:11.413711 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:11.413717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:11.414017 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 06:56:11.414232 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 06:56:11.414604 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":309,"threads_started":1}
I20260811 06:56:11.417251 11470 heartbeater.cc:347] Connected to a master server at 127.4.40.62:44223
I20260811 06:56:11.417580 11470 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:11.418455 11470 heartbeater.cc:511] Master 127.4.40.62:44223 requested a full tablet report, sending...
I20260811 06:56:11.420007 10880 ts_manager.cc:194] Registered new tserver with Master: f9fa38e15a544d3da90e02176dc74879 (127.4.40.2:34209)
I20260811 06:56:11.420868 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:57563
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:56:11.459033 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44223
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:11.801853 11480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:11.802155 11480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:11.802338 11480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:11.810002 11480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:11.810159 11480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:11.810364 11480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:11.820798 11480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44223
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:11.822120 11480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:11.823451 11480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:11.837294 11497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:11.837335 11488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:11.837529 11493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:11.838344 11480 server_base.cc:1034] running on GCE node
I20260811 06:56:11.839000 11480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:11.839932 11480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:11.841133 11480 hybrid_clock.cc:648] HybridClock initialized: now 1786431371841063 us; error 55 us; skew 500 ppm
I20260811 06:56:11.841423 11480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:11 dist-test-slave-7vjv krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:11.847731 11480 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:56:11.849226 11480 webserver.cc:460] Webserver started at http://127.4.40.3:33533/ using document root <none> and password file <none>
I20260811 06:56:11.849879 11480 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:11.850018 11480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:11.850314 11480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:11.853219 11480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "3994640811374adfbb2dc203f81a7d7b"
format_stamp: "Formatted at 2026-08-11 06:56:11 on dist-test-slave-7vjv"
server_key: "01c0519af74a92d54a5de1dfcf25d354"
server_key_iv: "53ee82a0652b92092f00cf608ad1a377"
server_key_version: "encryptionkey@0"
I20260811 06:56:11.853967 11480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "3994640811374adfbb2dc203f81a7d7b"
format_stamp: "Formatted at 2026-08-11 06:56:11 on dist-test-slave-7vjv"
server_key: "01c0519af74a92d54a5de1dfcf25d354"
server_key_iv: "53ee82a0652b92092f00cf608ad1a377"
server_key_version: "encryptionkey@0"
I20260811 06:56:11.858939 11480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 06:56:11.862659 11510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:11.863898 11480 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 06:56:11.864079 11480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "3994640811374adfbb2dc203f81a7d7b"
format_stamp: "Formatted at 2026-08-11 06:56:11 on dist-test-slave-7vjv"
server_key: "01c0519af74a92d54a5de1dfcf25d354"
server_key_iv: "53ee82a0652b92092f00cf608ad1a377"
server_key_version: "encryptionkey@0"
I20260811 06:56:11.864326 11480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:11.890476 11480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:11.894477 11480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:11.894780 11480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:11.895764 11480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:11.897112 11480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:11.897185 11480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:11.897265 11480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:11.897310 11480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:11.919106 11204 heartbeater.cc:503] Master 127.4.40.62:44223 was elected leader, sending a full tablet report...
I20260811 06:56:11.931597 11480 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:33725
I20260811 06:56:11.931671 11703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:33725 every 8 connection(s)
I20260811 06:56:11.933043 11480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:11.940718 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11480
I20260811 06:56:11.940927 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:11.941339 4256 external_mini_cluster.cc:1428] Setting key 2bea7bb0dd60b8ff6077cbf5e50ff97e
Aug 11 06:56:11 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:11.952435 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:11.935681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:41723 (local address 127.4.40.62:44223)
0811 06:56:11.936222 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:11.936227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:11.937050 (+ 823us) server_negotiation.cc:407] Connection header received
0811 06:56:11.938887 (+ 1837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:11.938893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:11.939007 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:11.939133 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:11.940679 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.941624 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.943446 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.943931 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.944079 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:11.947452 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:11.947456 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:11.947459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:11.947491 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:11.949942 (+ 2451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:11.950562 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:11.950565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:11.950567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:11.950682 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:11.951035 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:11.951038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:11.951040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:11.951273 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:11.951392 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:11.951953 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 06:56:11.952159 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":310,"threads_started":1}
I20260811 06:56:11.953323 11707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:11.936211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44223 (local address 127.4.40.3:41723)
0811 06:56:11.936898 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:11.936935 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:11.938605 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:11.939356 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:11.939364 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:11.939913 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:11.940516 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:11.940533 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.941787 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.941791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:11.943307 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:11.943319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:11.944193 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:11.944202 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:11.944412 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:11.945096 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:11.945117 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:11.947259 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:11.950059 (+ 2800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:11.950065 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:11.950076 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:11.950453 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:11.950766 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:11.950769 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:11.950771 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:11.950936 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:11.951386 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:11.951391 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:11.951651 (+ 260us) client_negotiation.cc:769] Sending connection context
0811 06:56:11.951856 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 06:56:11.952253 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":265,"threads_started":1}
I20260811 06:56:11.954671 11704 heartbeater.cc:347] Connected to a master server at 127.4.40.62:44223
I20260811 06:56:11.954999 11704 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:11.955840 11704 heartbeater.cc:511] Master 127.4.40.62:44223 requested a full tablet report, sending...
I20260811 06:56:11.957345 10880 ts_manager.cc:194] Registered new tserver with Master: 3994640811374adfbb2dc203f81a7d7b (127.4.40.3:33725)
I20260811 06:56:11.958174 10880 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:41723
I20260811 06:56:11.968529 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:56:11 dist-test-slave-7vjv krb5kdc[10802](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:12.037418 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:12.009864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49686 (local address 127.4.40.1:35991)
0811 06:56:12.010529 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:12.010537 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:12.010563 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:56:12.010626 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:12.010633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:12.010917 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:12.011068 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:12.012505 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.013656 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.014805 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.015439 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.015533 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:12.032001 (+ 16468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:12.032023 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:12.032035 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:12.032071 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:12.034825 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.035415 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.035422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.035424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.035546 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.035869 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.035872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.035874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.036298 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:12.036541 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:12.036779 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 06:56:12.036975 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":329,"threads_started":1}
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:12.054901 11706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:12.041880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.4.40.62:44223)
0811 06:56:12.042114 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:12.042121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:12.042350 (+ 229us) server_negotiation.cc:407] Connection header received
0811 06:56:12.042551 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:12.042555 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:12.042640 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:12.042720 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:12.044013 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.044799 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.045947 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.046312 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.046398 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:12.050096 (+ 3698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:12.050101 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:12.050104 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:12.050134 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:12.052740 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.053275 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.053278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.053280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.053387 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.053662 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.053665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.053667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.053950 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:12.054127 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:12.054380 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 06:56:12.054535 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 06:56:12.055516 10880 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37212
I20260811 06:56:12.056546 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10988
I20260811 06:56:12.068825 11716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:12.057400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49688 (local address 127.4.40.1:35991)
0811 06:56:12.057580 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:12.057588 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:12.057872 (+ 284us) server_negotiation.cc:407] Connection header received
0811 06:56:12.058062 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:12.058065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:12.058156 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:12.058259 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:12.059422 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.060178 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.061330 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.061726 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.061854 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:12.063910 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:12.063915 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:12.063918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:12.063948 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:12.066323 (+ 2375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.066933 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.066938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.066941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.067069 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.067411 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.067416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.067420 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.067681 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:12.067928 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:12.068305 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 06:56:12.068530 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 06:56:12.069382 11198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:12.153779 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10988
I20260811 06:56:12.185983 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11215
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:56:12.202713 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:12.187138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.4.40.2:34209)
0811 06:56:12.187721 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:12.187729 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:12.187754 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:56:12.187815 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:12.187821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:12.188047 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:12.188273 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:12.189529 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.190558 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.191815 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.192443 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.192586 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:12.196824 (+ 4238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:12.196835 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:12.196850 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:12.196880 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:12.199913 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.200606 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.200612 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.200615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.200735 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.201106 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.201111 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.201114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.201441 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:12.201795 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:12.202122 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 06:56:12.202378 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":303,"threads_started":1}
I20260811 06:56:12.204133 11463 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:12.288928 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11215
I20260811 06:56:12.320349 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11480
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[10802](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431371, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:56:12.342725 11729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:12.321393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54026 (local address 127.4.40.3:33725)
0811 06:56:12.322050 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:12.322056 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:12.322085 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:56:12.322146 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:12.322152 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:12.322358 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:12.322530 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:12.323862 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.324973 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.326281 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:12.326835 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:12.326982 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:12.332577 (+ 5595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:12.332589 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:12.332602 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:12.332633 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:12.336717 (+ 4084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.339933 (+ 3216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.339942 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.339946 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.340161 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:12.340677 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:12.340682 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:12.340685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:12.341160 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:12.341527 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:12.341890 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 06:56:12.342207 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":328,"threads_started":1}
I20260811 06:56:12.344313 11694 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:12.435262 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11480
I20260811 06:56:12.463097 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10821
I20260811 06:56:12.464341 10937 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:12.569796 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10821
2026-08-11T06:56:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4339 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[11741](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[11741](info): set up 2 sockets
Aug 11 06:56:12 dist-test-slave-7vjv krb5kdc[11741](info): commencing operation
krb5kdc: starting...
W20260811 06:56:13.914666 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.290s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:14 dist-test-slave-7vjv krb5kdc[11741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:14.151578 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46277
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:46277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:14.497057 11763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:14.497406 11763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:14.497586 11763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:14.505095 11763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:14.505198 11763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:14.505293 11763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:14.505389 11763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:14.505470 11763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:14.515208 11763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:46277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:14.516537 11763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:14.517863 11763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:14.529573 11773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:14.529798 11769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:14.529666 11771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:14.530797 11763 server_base.cc:1034] running on GCE node
I20260811 06:56:14.531538 11763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:14.532990 11763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:14.534211 11763 hybrid_clock.cc:648] HybridClock initialized: now 1786431374534168 us; error 38 us; skew 500 ppm
I20260811 06:56:14.534570 11763 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:14 dist-test-slave-7vjv krb5kdc[11741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431374, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:14.537737 11763 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:14.538865 11763 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:14.538976 11763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:14.539309 11763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:14.542333 11763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "f0a249f8ddf24ddbb73c61ff47c068c3"
format_stamp: "Formatted at 2026-08-11 06:56:14 on dist-test-slave-7vjv"
server_key: "f0195716acd37cce26a7ead093340bf9"
server_key_iv: "f10bc9532bb9e2164fefe3a75b989579"
server_key_version: "encryptionkey@0"
I20260811 06:56:14.543251 11763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "f0a249f8ddf24ddbb73c61ff47c068c3"
format_stamp: "Formatted at 2026-08-11 06:56:14 on dist-test-slave-7vjv"
server_key: "f0195716acd37cce26a7ead093340bf9"
server_key_iv: "f10bc9532bb9e2164fefe3a75b989579"
server_key_version: "encryptionkey@0"
I20260811 06:56:14.548236 11763 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 06:56:14.551508 11782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:14.552809 11763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:14.552986 11763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "f0a249f8ddf24ddbb73c61ff47c068c3"
format_stamp: "Formatted at 2026-08-11 06:56:14 on dist-test-slave-7vjv"
server_key: "f0195716acd37cce26a7ead093340bf9"
server_key_iv: "f10bc9532bb9e2164fefe3a75b989579"
server_key_version: "encryptionkey@0"
I20260811 06:56:14.553205 11763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:14.583331 11763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:14.587491 11763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:14.587795 11763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:14.606701 11763 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:46277
I20260811 06:56:14.606673 11869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:46277 every 8 connection(s)
I20260811 06:56:14.607612 11763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:14.611624 11870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:14.614540 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11763
I20260811 06:56:14.614784 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:14.615608 4256 external_mini_cluster.cc:1428] Setting key da337d3c86f956e40c8dc0fab91e21d3
I20260811 06:56:14.624408 11870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3: Bootstrap starting.
I20260811 06:56:14.628088 11870 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:14.629412 11870 log.cc:824] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:56:14 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:14.632011 11870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3: No bootstrap required, opened a new log
I20260811 06:56:14.638728 11874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:14.617817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.4.40.62:46277)
0811 06:56:14.618601 (+ 784us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:14.618612 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:14.618652 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:56:14.620441 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:14.620462 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:14.620839 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:14.621145 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:14.622988 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:14.624344 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:14.626227 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:14.626872 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:14.626990 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:14.631589 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:14.631596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:14.631608 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:14.631643 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:14.634703 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:14.635375 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:14.635382 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:14.635388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:14.635556 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:14.635927 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:14.635932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:14.635943 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:14.636561 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:14.636899 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:14.637330 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 06:56:14.637768 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":388,"threads_started":1}
I20260811 06:56:14.639987 11870 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46277 } }
I20260811 06:56:14.640491 11870 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:14.640605 11870 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a249f8ddf24ddbb73c61ff47c068c3, State: Initialized, Role: FOLLOWER
I20260811 06:56:14.641219 11870 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46277 } }
I20260811 06:56:14.641539 11870 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:14.641739 11870 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:14.641908 11870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:14.643939 11870 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46277 } }
I20260811 06:56:14.644428 11870 leader_election.cc:302] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0a249f8ddf24ddbb73c61ff47c068c3; no voters:
I20260811 06:56:14.645159 11870 leader_election.cc:288] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:14.645255 11876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:14.646021 11876 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [term 1 LEADER]: Becoming Leader. State: Replica: f0a249f8ddf24ddbb73c61ff47c068c3, State: Running, Role: LEADER
I20260811 06:56:14.646562 11876 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46277 } }
I20260811 06:56:14.647423 11870 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:14.649549 11879 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0a249f8ddf24ddbb73c61ff47c068c3. Latest consensus state: current_term: 1 leader_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46277 } } }
I20260811 06:56:14.649780 11878 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0a249f8ddf24ddbb73c61ff47c068c3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46277 } } }
I20260811 06:56:14.650095 11878 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:14.650104 11879 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:14.657251 11886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:14.662844 11886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:14.671718 11886 catalog_manager.cc:1347] Generated new cluster ID: a49322119dfb4311b68274da47123c7f
I20260811 06:56:14.671835 11886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:14.684895 11886 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:14.686126 11886 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:14.691561 11886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3: Generated new TSK 0
I20260811 06:56:14.692552 11886 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:56:14.750252 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46277
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:15.103895 11909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:15.104338 11909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:15.104524 11909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:15.112581 11909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:15.112710 11909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:15.112885 11909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:15.123932 11909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:15.125474 11909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:15.127533 11909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:15.143512 11915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:15.143548 11919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:15.143482 11917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:15.144271 11909 server_base.cc:1034] running on GCE node
I20260811 06:56:15.144904 11909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:15.145813 11909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:15.147019 11909 hybrid_clock.cc:648] HybridClock initialized: now 1786431375146980 us; error 29 us; skew 500 ppm
I20260811 06:56:15.147395 11909 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:15 dist-test-slave-7vjv krb5kdc[11741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431375, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:15.150547 11909 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:56:15.151662 11909 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:15.151793 11909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:15.152175 11909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:15.155036 11909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "314a2552c9544a20b7925cc9002c01ca"
format_stamp: "Formatted at 2026-08-11 06:56:15 on dist-test-slave-7vjv"
server_key: "8b54e0ae37a42edfeffcccfc52672569"
server_key_iv: "0054897873ea686009aa1935f4101a19"
server_key_version: "encryptionkey@0"
I20260811 06:56:15.155880 11909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "314a2552c9544a20b7925cc9002c01ca"
format_stamp: "Formatted at 2026-08-11 06:56:15 on dist-test-slave-7vjv"
server_key: "8b54e0ae37a42edfeffcccfc52672569"
server_key_iv: "0054897873ea686009aa1935f4101a19"
server_key_version: "encryptionkey@0"
I20260811 06:56:15.161206 11909 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:56:15.164590 11935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:15.165822 11909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:56:15.165994 11909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "314a2552c9544a20b7925cc9002c01ca"
format_stamp: "Formatted at 2026-08-11 06:56:15 on dist-test-slave-7vjv"
server_key: "8b54e0ae37a42edfeffcccfc52672569"
server_key_iv: "0054897873ea686009aa1935f4101a19"
server_key_version: "encryptionkey@0"
I20260811 06:56:15.166220 11909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:15.180692 11909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:15.184348 11909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:15.184625 11909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:15.185523 11909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:15.186776 11909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:15.186902 11909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:15.187023 11909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:15.187114 11909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:15.220932 11909 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:34879
I20260811 06:56:15.220993 12129 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:34879 every 8 connection(s)
I20260811 06:56:15.221820 11909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:15.226987 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11909
I20260811 06:56:15.227240 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:15.227885 4256 external_mini_cluster.cc:1428] Setting key a17eca841d8e04f5c5d6e6d6784d0f43
Aug 11 06:56:15 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431375, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:15.241777 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:15.224286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:44953 (local address 127.4.40.62:46277)
0811 06:56:15.224765 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:15.224769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:15.225627 (+ 858us) server_negotiation.cc:407] Connection header received
0811 06:56:15.227572 (+ 1945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:15.227578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:15.227679 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:15.227809 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:15.229511 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.230245 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.231780 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.232226 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.232350 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:15.235926 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:15.235931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:15.235935 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:15.235966 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:15.238553 (+ 2587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:15.239296 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:15.239303 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:15.239306 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:15.239432 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:15.239931 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:15.239937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:15.239940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:15.240285 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:15.240451 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:15.241154 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 06:56:15.241384 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":258,"threads_started":1}
I20260811 06:56:15.242743 12138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:15.224758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46277 (local address 127.4.40.1:44953)
0811 06:56:15.225465 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:15.225504 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:15.227268 (+ 1764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:15.228057 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:15.228064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:15.228697 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:15.229340 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:15.229358 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.230400 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.230405 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:15.231628 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:15.231640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.232464 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.232472 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:15.232683 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:15.233347 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:15.233372 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:15.235714 (+ 2342us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:15.238691 (+ 2977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:15.238700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:15.238715 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:15.239178 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:15.239571 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:15.239585 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:15.239589 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:15.239815 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:15.240414 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:15.240423 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:15.240760 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 06:56:15.241059 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 06:56:15.241596 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":298,"threads_started":1}
I20260811 06:56:15.244475 12131 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46277
I20260811 06:56:15.244815 12131 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:15.245635 12131 heartbeater.cc:511] Master 127.4.40.62:46277 requested a full tablet report, sending...
I20260811 06:56:15.248289 11807 ts_manager.cc:194] Registered new tserver with Master: 314a2552c9544a20b7925cc9002c01ca (127.4.40.1:34879)
I20260811 06:56:15.250564 11807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:44953
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:56:15.286654 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46277
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:15.618969 12143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:15.619232 12143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:15.619403 12143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:15.626741 12143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:15.626878 12143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:15.627063 12143 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:15.636922 12143 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:15.638331 12143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:15.639662 12143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:15.652631 12156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:15.652598 12160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:15.652676 12155 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:15.657775 12143 server_base.cc:1034] running on GCE node
I20260811 06:56:15.658497 12143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:15.659420 12143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:15.660624 12143 hybrid_clock.cc:648] HybridClock initialized: now 1786431375660580 us; error 40 us; skew 500 ppm
I20260811 06:56:15.660938 12143 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:15 dist-test-slave-7vjv krb5kdc[11741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431375, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:15.664350 12143 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:56:15.665562 12143 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:15.665655 12143 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:15.665916 12143 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:15.668941 12143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "c15aaf21121e471eb9905cedb40feac4"
format_stamp: "Formatted at 2026-08-11 06:56:15 on dist-test-slave-7vjv"
server_key: "81bb696a91324aaaabb4c99df3f23181"
server_key_iv: "7dc8ad056347390eef282aa36fd54259"
server_key_version: "encryptionkey@0"
I20260811 06:56:15.669778 12143 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "c15aaf21121e471eb9905cedb40feac4"
format_stamp: "Formatted at 2026-08-11 06:56:15 on dist-test-slave-7vjv"
server_key: "81bb696a91324aaaabb4c99df3f23181"
server_key_iv: "7dc8ad056347390eef282aa36fd54259"
server_key_version: "encryptionkey@0"
I20260811 06:56:15.674821 12143 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 06:56:15.678130 12169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:15.679446 12143 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:56:15.679628 12143 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "c15aaf21121e471eb9905cedb40feac4"
format_stamp: "Formatted at 2026-08-11 06:56:15 on dist-test-slave-7vjv"
server_key: "81bb696a91324aaaabb4c99df3f23181"
server_key_iv: "7dc8ad056347390eef282aa36fd54259"
server_key_version: "encryptionkey@0"
I20260811 06:56:15.679843 12143 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:15.690990 12143 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:15.695313 12143 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:15.695597 12143 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:15.696615 12143 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:15.698002 12143 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:15.698123 12143 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:15.698253 12143 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:15.698340 12143 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:15.732556 12143 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:34527
I20260811 06:56:15.732640 12378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:34527 every 8 connection(s)
I20260811 06:56:15.733439 12143 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:15.739611 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12143
I20260811 06:56:15.739827 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:15.740351 4256 external_mini_cluster.cc:1428] Setting key ab914340bb186080819ee3b7d9d81bab
Aug 11 06:56:15 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431375, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:15.753634 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:15.735900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:38741 (local address 127.4.40.62:46277)
0811 06:56:15.736084 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:15.736119 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:15.737317 (+ 1198us) server_negotiation.cc:407] Connection header received
0811 06:56:15.739142 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:15.739148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:15.739239 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:15.739365 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:15.741053 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.741781 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.743331 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.743876 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.744001 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:15.747652 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:15.747658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:15.747661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:15.747695 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:15.750577 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:15.751364 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:15.751369 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:15.751378 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:15.751509 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:15.751938 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:15.751942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:15.751945 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:15.752262 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:15.752413 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:15.753073 (+ 660us) server_negotiation.cc:299] Negotiation successful
0811 06:56:15.753295 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:56:15.754621 12382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:15.736411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46277 (local address 127.4.40.2:38741)
0811 06:56:15.737130 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:15.737177 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:15.738871 (+ 1694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:15.739614 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:15.739623 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:15.740266 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:15.740872 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:15.740889 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.741942 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.741946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:15.743186 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:15.743199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:15.744133 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:15.744141 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:15.744352 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:15.745034 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:15.745059 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:15.747413 (+ 2354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:15.750739 (+ 3326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:15.750748 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:15.750763 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:15.751211 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:15.751620 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:15.751624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:15.751628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:15.751821 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:15.752386 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:15.752394 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:15.752712 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 06:56:15.752972 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 06:56:15.753457 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":300,"threads_started":1}
I20260811 06:56:15.756249 12379 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46277
I20260811 06:56:15.756608 12379 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:15.757525 12379 heartbeater.cc:511] Master 127.4.40.62:46277 requested a full tablet report, sending...
I20260811 06:56:15.759430 11807 ts_manager.cc:194] Registered new tserver with Master: c15aaf21121e471eb9905cedb40feac4 (127.4.40.2:34527)
I20260811 06:56:15.760895 11807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:38741
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:56:15.799119 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46277
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:16.129779 12388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:16.130040 12388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:16.130220 12388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:16.137300 12388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:16.137426 12388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:16.137606 12388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:16.147243 12388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46277
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:16.148610 12388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:16.149923 12388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:16.162613 12398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:16.162562 12401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:16.162542 12405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:16.163626 12388 server_base.cc:1034] running on GCE node
I20260811 06:56:16.164240 12388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:16.165139 12388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:16.166324 12388 hybrid_clock.cc:648] HybridClock initialized: now 1786431376166270 us; error 49 us; skew 500 ppm
I20260811 06:56:16.166627 12388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:16 dist-test-slave-7vjv krb5kdc[11741](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:16.170061 12388 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:56:16.171293 12388 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:16.171396 12388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:16.171674 12388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:16.174587 12388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "c79eb16e507f4025a37189987f935b31"
format_stamp: "Formatted at 2026-08-11 06:56:16 on dist-test-slave-7vjv"
server_key: "7811bed53146db10012da005797c1b67"
server_key_iv: "20e92249c0ef1b5f3f1ec55f8d087185"
server_key_version: "encryptionkey@0"
I20260811 06:56:16.175374 12388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "c79eb16e507f4025a37189987f935b31"
format_stamp: "Formatted at 2026-08-11 06:56:16 on dist-test-slave-7vjv"
server_key: "7811bed53146db10012da005797c1b67"
server_key_iv: "20e92249c0ef1b5f3f1ec55f8d087185"
server_key_version: "encryptionkey@0"
I20260811 06:56:16.180047 12388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.006s
I20260811 06:56:16.183267 12418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:16.184705 12388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:16.184881 12388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "c79eb16e507f4025a37189987f935b31"
format_stamp: "Formatted at 2026-08-11 06:56:16 on dist-test-slave-7vjv"
server_key: "7811bed53146db10012da005797c1b67"
server_key_iv: "20e92249c0ef1b5f3f1ec55f8d087185"
server_key_version: "encryptionkey@0"
I20260811 06:56:16.185093 12388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:16.205453 12388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:16.209976 12388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:16.210285 12388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:16.211297 12388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:16.212726 12388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:16.212850 12388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:16.212991 12388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:16.213081 12388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:16.248267 12388 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:40437
I20260811 06:56:16.248327 12622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:40437 every 8 connection(s)
I20260811 06:56:16.249217 12388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:16.251941 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12388
I20260811 06:56:16.252192 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:16.252786 4256 external_mini_cluster.cc:1428] Setting key 523b94ff1b6cf13a2b078a2f5356314d
I20260811 06:56:16.254338 12131 heartbeater.cc:503] Master 127.4.40.62:46277 was elected leader, sending a full tablet report...
Aug 11 06:56:16 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431376, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:16.269375 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.251659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:52081 (local address 127.4.40.62:46277)
0811 06:56:16.251890 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.251896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.253130 (+ 1234us) server_negotiation.cc:407] Connection header received
0811 06:56:16.255646 (+ 2516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.255651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.255736 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.255834 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:16.257706 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.258585 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.260039 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.260513 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.260610 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.263720 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:16.263725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:16.263728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:16.263759 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:16.266416 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.267191 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.267200 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.267203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.267329 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.267744 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.267747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.267749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.268138 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:16.268259 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.268854 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.269048 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:56:16.270288 12627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.252028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46277 (local address 127.4.40.3:52081)
0811 06:56:16.252935 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.253008 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:16.255290 (+ 2282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:16.256362 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:16.256369 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:16.256878 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:16.257530 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.257546 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.258714 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.258718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.259906 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.259918 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.260695 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.260700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.260891 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.261494 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:16.261521 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:16.263492 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:16.266557 (+ 3065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:16.266563 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:16.266574 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:16.267054 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:16.267440 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:16.267444 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:16.267448 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:16.267651 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:16.268271 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:16.268277 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:16.268543 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 06:56:16.268754 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 06:56:16.269169 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"thread_start_us":359,"threads_started":1}
I20260811 06:56:16.271620 12623 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46277
I20260811 06:56:16.271896 12623 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:16.272550 12623 heartbeater.cc:511] Master 127.4.40.62:46277 requested a full tablet report, sending...
I20260811 06:56:16.273857 11807 ts_manager.cc:194] Registered new tserver with Master: c79eb16e507f4025a37189987f935b31 (127.4.40.3:40437)
I20260811 06:56:16.274695 11807 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:52081
I20260811 06:56:16.280058 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:56:16.296567 12136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.284551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.4.40.62:46277)
0811 06:56:16.284690 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.284694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.285213 (+ 519us) server_negotiation.cc:407] Connection header received
0811 06:56:16.285334 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.285338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.285410 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.285489 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:16.286892 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.287557 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.288808 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.289268 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.289360 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.291695 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:16.291699 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:16.291702 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:16.291730 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:16.294113 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.294802 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.294805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.294808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.294921 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.295230 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.295233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.295235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.295541 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:16.295735 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.296024 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.296241 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:56:16.300599 11807 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:56:16.307549 11807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:16.331578 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.320311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.4.40.1:34879)
0811 06:56:16.322879 (+ 2568us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.322890 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.322923 (+ 33us) server_negotiation.cc:407] Connection header received
0811 06:56:16.323028 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.323037 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.323261 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.323464 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:16.324580 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.325875 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.328546 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.329786 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.329931 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.330169 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.331039 (+ 870us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.331207 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2364,"thread_start_us":2196,"threads_started":1}
I20260811 06:56:16.331784 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.319760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.4.40.2:34527)
0811 06:56:16.320903 (+ 1143us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.320915 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.321822 (+ 907us) server_negotiation.cc:407] Connection header received
0811 06:56:16.322318 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.322330 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.322972 (+ 642us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.323202 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:16.324277 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.325872 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.329158 (+ 3286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.330604 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.330733 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.330902 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.331237 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.331409 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":825,"threads_started":1}
I20260811 06:56:16.332149 12649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.319670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41156)
0811 06:56:16.322093 (+ 2423us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.322112 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:16.322228 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:16.323705 (+ 1477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:16.323709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:16.323738 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:16.324347 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.324361 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.326079 (+ 1718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.326087 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.328368 (+ 2281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.328383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.330306 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.330314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.330510 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.330564 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 06:56:16.330698 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 06:56:16.330896 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2302,"mutex_wait_us":44,"spinlock_wait_cycles":3968,"thread_start_us":3438,"threads_started":1}
I20260811 06:56:16.332696 12648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.319454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:34527 (local address 127.0.0.1:45290)
0811 06:56:16.321635 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.321695 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:16.322025 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:16.323523 (+ 1498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:16.323531 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:16.323567 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:16.324068 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.324079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.326003 (+ 1924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.326008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.328978 (+ 2970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.328993 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.330823 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.330830 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.330991 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.331046 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 06:56:16.332068 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0811 06:56:16.332222 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1853,"mutex_wait_us":88,"thread_start_us":1669,"threads_started":1}
I20260811 06:56:16.333123 12654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.321498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:40437 (local address 127.0.0.1:54332)
0811 06:56:16.323753 (+ 2255us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.323770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:16.323889 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:16.325570 (+ 1681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:16.325574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:16.325600 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:16.325945 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.325961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.328018 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.328024 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.329779 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.329790 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.331631 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.331637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.332326 (+ 689us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.332370 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 06:56:16.332475 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:56:16.332616 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2098,"thread_start_us":1955,"threads_started":1}
I20260811 06:56:16.333388 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.322137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.4.40.3:40437)
0811 06:56:16.325031 (+ 2894us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.325039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.325070 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:56:16.325173 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.325180 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.325379 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.325561 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:16.326172 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.327833 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.329929 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.331133 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.331252 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.331467 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.332709 (+ 1242us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.333023 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2732,"thread_start_us":2527,"threads_started":1}
I20260811 06:56:16.336400 12019 tablet_service.cc:1467] Processing CreateTablet for tablet e4c384d8e7df489990954a423a95055f (DEFAULT_TABLE table=test-table [id=45cf10435d344af7a7d26b744805fd44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:16.336421 12259 tablet_service.cc:1467] Processing CreateTablet for tablet e4c384d8e7df489990954a423a95055f (DEFAULT_TABLE table=test-table [id=45cf10435d344af7a7d26b744805fd44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:16.337044 12512 tablet_service.cc:1467] Processing CreateTablet for tablet e4c384d8e7df489990954a423a95055f (DEFAULT_TABLE table=test-table [id=45cf10435d344af7a7d26b744805fd44]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:16.337963 12019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c384d8e7df489990954a423a95055f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:16.338335 12512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c384d8e7df489990954a423a95055f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:16.339118 12259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c384d8e7df489990954a423a95055f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:16.352035 12658 tablet_bootstrap.cc:492] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Bootstrap starting.
I20260811 06:56:16.354768 12661 tablet_bootstrap.cc:492] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Bootstrap starting.
I20260811 06:56:16.355938 12658 tablet_bootstrap.cc:654] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:16.356655 12659 tablet_bootstrap.cc:492] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Bootstrap starting.
I20260811 06:56:16.357502 12658 log.cc:824] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:16.359316 12661 tablet_bootstrap.cc:654] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:16.360569 12659 tablet_bootstrap.cc:654] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:16.360620 12661 log.cc:824] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:16.361498 12658 tablet_bootstrap.cc:492] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: No bootstrap required, opened a new log
I20260811 06:56:16.361690 12659 log.cc:824] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:16.361763 12658 ts_tablet_manager.cc:1397] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:56:16.363654 12661 tablet_bootstrap.cc:492] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: No bootstrap required, opened a new log
I20260811 06:56:16.363903 12661 ts_tablet_manager.cc:1397] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 06:56:16.364194 12659 tablet_bootstrap.cc:492] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: No bootstrap required, opened a new log
I20260811 06:56:16.364439 12659 ts_tablet_manager.cc:1397] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 06:56:16.371814 12659 raft_consensus.cc:348] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.372206 12659 raft_consensus.cc:374] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:16.372313 12659 raft_consensus.cc:729] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 314a2552c9544a20b7925cc9002c01ca, State: Initialized, Role: FOLLOWER
I20260811 06:56:16.372942 12659 consensus_queue.cc:260] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.372828 12658 raft_consensus.cc:348] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.373227 12658 raft_consensus.cc:374] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:16.373332 12658 raft_consensus.cc:729] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c79eb16e507f4025a37189987f935b31, State: Initialized, Role: FOLLOWER
I20260811 06:56:16.373920 12658 consensus_queue.cc:260] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.374349 12661 raft_consensus.cc:348] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.374788 12661 raft_consensus.cc:374] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:16.374890 12661 raft_consensus.cc:729] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c15aaf21121e471eb9905cedb40feac4, State: Initialized, Role: FOLLOWER
I20260811 06:56:16.375308 12623 heartbeater.cc:503] Master 127.4.40.62:46277 was elected leader, sending a full tablet report...
I20260811 06:56:16.375453 12661 consensus_queue.cc:260] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.376667 12379 heartbeater.cc:503] Master 127.4.40.62:46277 was elected leader, sending a full tablet report...
I20260811 06:56:16.377290 12661 ts_tablet_manager.cc:1428] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260811 06:56:16.378031 12658 ts_tablet_manager.cc:1428] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Time spent starting tablet: real 0.016s user 0.006s sys 0.009s
I20260811 06:56:16.381155 12659 ts_tablet_manager.cc:1428] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Time spent starting tablet: real 0.017s user 0.002s sys 0.013s
W20260811 06:56:16.474126 12133 tablet.cc:2369] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:56:16.485363 12380 tablet.cc:2369] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:56:16.500653 12625 tablet.cc:2369] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:16.621728 12667 raft_consensus.cc:482] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:16.622047 12667 raft_consensus.cc:504] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.623992 12667 leader_election.cc:288] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c15aaf21121e471eb9905cedb40feac4 (127.4.40.2:34527), c79eb16e507f4025a37189987f935b31 (127.4.40.3:40437)
I20260811 06:56:16.632990 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.624498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:40827 (local address 127.4.40.3:40437)
0811 06:56:16.624722 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.624729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.625391 (+ 662us) server_negotiation.cc:407] Connection header received
0811 06:56:16.625609 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.625614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.625690 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.625792 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:16.626545 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.627685 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.630782 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.632059 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.632221 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.632327 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.632572 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.632716 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:56:16.633023 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.624324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:40437 (local address 127.4.40.1:40827)
0811 06:56:16.625253 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.625276 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:16.625475 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:16.625825 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:16.625830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:16.625860 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:16.626367 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.626388 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.627822 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.627827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.630604 (+ 2777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.630622 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.632271 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.632276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.632397 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.632422 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:56:16.632518 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:56:16.632613 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"mutex_wait_us":154,"spinlock_wait_cycles":20864,"thread_start_us":481,"threads_started":1}
I20260811 06:56:16.633586 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.624398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:34527 (local address 127.4.40.1:57885)
0811 06:56:16.625260 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.625276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:16.625434 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:16.625825 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:16.625830 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:16.625886 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:16.626437 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.626450 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.627822 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.627827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.630611 (+ 2784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:16.630622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.632530 (+ 1908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.632539 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:16.632727 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.632758 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:56:16.632866 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:56:16.632995 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":760,"mutex_wait_us":186,"spinlock_wait_cycles":30080,"thread_start_us":534,"threads_started":1}
I20260811 06:56:16.633631 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.624579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57885 (local address 127.4.40.2:34527)
0811 06:56:16.624916 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.624922 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.625385 (+ 463us) server_negotiation.cc:407] Connection header received
0811 06:56:16.625604 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.625609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.625690 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.625793 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:16.626598 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.627684 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.630782 (+ 3098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.632046 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.632221 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.632322 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.632920 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.633101 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260811 06:56:16.634217 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c384d8e7df489990954a423a95055f" candidate_uuid: "314a2552c9544a20b7925cc9002c01ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79eb16e507f4025a37189987f935b31" is_pre_election: true
I20260811 06:56:16.634552 12294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c384d8e7df489990954a423a95055f" candidate_uuid: "314a2552c9544a20b7925cc9002c01ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15aaf21121e471eb9905cedb40feac4" is_pre_election: true
I20260811 06:56:16.634723 12541 raft_consensus.cc:2454] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 314a2552c9544a20b7925cc9002c01ca in term 0.
I20260811 06:56:16.635063 12294 raft_consensus.cc:2454] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 314a2552c9544a20b7925cc9002c01ca in term 0.
I20260811 06:56:16.635478 11939 leader_election.cc:302] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 314a2552c9544a20b7925cc9002c01ca, c79eb16e507f4025a37189987f935b31; no voters:
I20260811 06:56:16.635880 12667 raft_consensus.cc:2781] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:16.636045 12667 raft_consensus.cc:482] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:16.636193 12667 raft_consensus.cc:3037] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:16.638414 12667 raft_consensus.cc:504] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.639093 12667 leader_election.cc:288] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [CANDIDATE]: Term 1 election: Requested vote from peers c15aaf21121e471eb9905cedb40feac4 (127.4.40.2:34527), c79eb16e507f4025a37189987f935b31 (127.4.40.3:40437)
I20260811 06:56:16.639492 12294 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c384d8e7df489990954a423a95055f" candidate_uuid: "314a2552c9544a20b7925cc9002c01ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c15aaf21121e471eb9905cedb40feac4"
I20260811 06:56:16.639611 12541 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c384d8e7df489990954a423a95055f" candidate_uuid: "314a2552c9544a20b7925cc9002c01ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c79eb16e507f4025a37189987f935b31"
I20260811 06:56:16.639698 12294 raft_consensus.cc:3037] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:16.639838 12541 raft_consensus.cc:3037] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:16.642023 12294 raft_consensus.cc:2454] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 314a2552c9544a20b7925cc9002c01ca in term 1.
I20260811 06:56:16.642225 12541 raft_consensus.cc:2454] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 314a2552c9544a20b7925cc9002c01ca in term 1.
I20260811 06:56:16.642482 11941 leader_election.cc:302] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 314a2552c9544a20b7925cc9002c01ca, c15aaf21121e471eb9905cedb40feac4; no voters:
I20260811 06:56:16.642797 12667 raft_consensus.cc:2781] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:16.643241 12667 raft_consensus.cc:686] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 1 LEADER]: Becoming Leader. State: Replica: 314a2552c9544a20b7925cc9002c01ca, State: Running, Role: LEADER
I20260811 06:56:16.643750 12667 consensus_queue.cc:237] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } }
I20260811 06:56:16.648686 11807 catalog_manager.cc:5495] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 314a2552c9544a20b7925cc9002c01ca (127.4.40.1). New cstate: current_term: 1 leader_uuid: "314a2552c9544a20b7925cc9002c01ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "314a2552c9544a20b7925cc9002c01ca" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 } health_report { overall_health: UNKNOWN } } }
I20260811 06:56:16.695287 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.690347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41172 (local address 127.4.40.1:34879)
0811 06:56:16.690601 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.690605 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.690623 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:56:16.690805 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.690808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.690883 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.690994 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:16.691332 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.691997 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.693279 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.693695 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.693786 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.694199 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:16.694704 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:16.694808 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.694902 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.694966 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:56:16.703279 12541 raft_consensus.cc:1261] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 1 FOLLOWER]: Refusing update from remote peer 314a2552c9544a20b7925cc9002c01ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:16.703313 12294 raft_consensus.cc:1261] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 1 FOLLOWER]: Refusing update from remote peer 314a2552c9544a20b7925cc9002c01ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:16.704273 12676 consensus_queue.cc:1035] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "c79eb16e507f4025a37189987f935b31" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:16.704659 12667 consensus_queue.cc:1035] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "c15aaf21121e471eb9905cedb40feac4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:16.722084 12685 mvcc.cc:205] Tried to move back new op lower bound from 7317222918967824384 to 7317222918736912384. Current Snapshot: MvccSnapshot[applied={T|T < 7317222918967824384}]
I20260811 06:56:16.730635 11806 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:16.730953 11806 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40644:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:16.734957 11806 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f0a249f8ddf24ddbb73c61ff47c068c3: Sending DeleteTablet for 3 replicas of tablet e4c384d8e7df489990954a423a95055f
I20260811 06:56:16.736079 12019 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c384d8e7df489990954a423a95055f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:16 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:41156
I20260811 06:56:16.736150 12259 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c384d8e7df489990954a423a95055f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:16 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:45290
I20260811 06:56:16.739065 12512 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c384d8e7df489990954a423a95055f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:16 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:54332
I20260811 06:56:16.747629 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11909
I20260811 06:56:16.748600 12699 tablet_replica.cc:330] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: stopping tablet replica
I20260811 06:56:16.749229 12699 raft_consensus.cc:2229] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:16.749750 12699 raft_consensus.cc:2258] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:16.753999 12699 ts_tablet_manager.cc:1905] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:16.762552 12699 ts_tablet_manager.cc:1918] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:16.762825 12699 log.cc:1196] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/e4c384d8e7df489990954a423a95055f
I20260811 06:56:16.763634 12699 ts_tablet_manager.cc:1939] T e4c384d8e7df489990954a423a95055f P c15aaf21121e471eb9905cedb40feac4: Deleting consensus metadata
I20260811 06:56:16.765704 11786 catalog_manager.cc:4841] TS c15aaf21121e471eb9905cedb40feac4 (127.4.40.2:34527): tablet e4c384d8e7df489990954a423a95055f (table test-table [id=45cf10435d344af7a7d26b744805fd44]) successfully deleted
I20260811 06:56:16.767884 12702 tablet_replica.cc:330] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: stopping tablet replica
I20260811 06:56:16.769474 12702 raft_consensus.cc:2229] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:16.770145 12702 raft_consensus.cc:2258] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 06:56:16 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:16.774178 12697 tablet_replica.cc:330] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: stopping tablet replica
I20260811 06:56:16.774606 12697 raft_consensus.cc:2229] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:16.775357 12697 raft_consensus.cc:2258] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:16.775931 12702 ts_tablet_manager.cc:1905] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:16.777652 12697 ts_tablet_manager.cc:1905] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:16.782927 12702 ts_tablet_manager.cc:1918] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:16.783162 12702 log.cc:1196] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/e4c384d8e7df489990954a423a95055f
I20260811 06:56:16.783720 12702 ts_tablet_manager.cc:1939] T e4c384d8e7df489990954a423a95055f P c79eb16e507f4025a37189987f935b31: Deleting consensus metadata
I20260811 06:56:16.783748 12653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.748981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.4.40.1:34879)
0811 06:56:16.749153 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.749160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.754578 (+ 5418us) server_negotiation.cc:407] Connection header received
0811 06:56:16.754755 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.754759 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.754856 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.754986 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:16.764252 (+ 9266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.765462 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.767301 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.768045 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.768206 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.775361 (+ 7155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:16.775376 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:16.775392 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:16.775434 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:16.779867 (+ 4433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.780795 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.780804 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.780808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.780995 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.781498 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.781504 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.781507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.781971 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:16.782294 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.783053 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.783313 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 06:56:16.785362 12124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:16.785847 11785 catalog_manager.cc:4841] TS c79eb16e507f4025a37189987f935b31 (127.4.40.3:40437): tablet e4c384d8e7df489990954a423a95055f (table test-table [id=45cf10435d344af7a7d26b744805fd44]) successfully deleted
I20260811 06:56:16.885998 12697 ts_tablet_manager.cc:1918] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:16.886298 12697 log.cc:1196] T e4c384d8e7df489990954a423a95055f P 314a2552c9544a20b7925cc9002c01ca: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/e4c384d8e7df489990954a423a95055f
I20260811 06:56:16.886678 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11909
I20260811 06:56:16.916471 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12143
W20260811 06:56:16.916385 11783 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:34879 (error 108)
I20260811 06:56:16.918509 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.917786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41184)
0811 06:56:16.918235 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.918341 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":240,"threads_started":1}
W20260811 06:56:16.918766 11783 catalog_manager.cc:4584] TS 314a2552c9544a20b7925cc9002c01ca (127.4.40.1:34879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4c384d8e7df489990954a423a95055f: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Aug 11 06:56:16 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:56:16.933635 12650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.917366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45298 (local address 127.4.40.2:34527)
0811 06:56:16.917559 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:16.917567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:16.917607 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:56:16.917822 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:16.917826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:16.917889 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:16.918018 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:16.919450 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.920202 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.921389 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:16.921762 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:16.921848 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:16.928335 (+ 6487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:16.928350 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:16.928364 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:16.928395 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:16.931156 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.931781 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.931788 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.931790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.931921 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:16.932361 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:16.932364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:16.932366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:16.932744 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:16.932976 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:16.933181 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 06:56:16.933358 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:56:16.934244 12372 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:16.972504 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.972052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41192)
0811 06:56:16.972266 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.972369 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 06:56:16.973512 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:16.973174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41196)
0811 06:56:16.973313 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:16.973377 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 06:56:17.024123 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12143
I20260811 06:56:17.033804 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.033363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41202)
0811 06:56:17.033574 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:17.033657 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:56:17.034878 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.034522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41216)
0811 06:56:17.034676 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:17.034748 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:56:17.051345 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12388
Aug 11 06:56:17 dist-test-slave-7vjv krb5kdc[11741](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431374, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:56:17.066035 12657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.052156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54334 (local address 127.4.40.3:40437)
0811 06:56:17.052343 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:17.052348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:17.052367 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:56:17.052531 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:17.052536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:17.052601 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:17.052697 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:17.053916 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:17.054623 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:17.055792 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:17.056280 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:17.056373 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:17.060772 (+ 4399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:17.060787 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:17.060805 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:17.060837 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:17.063641 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:17.064263 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:17.064271 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:17.064273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:17.064403 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:17.064725 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:17.064728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:17.064730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:17.065067 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:17.065314 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:17.065505 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 06:56:17.065665 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:56:17.066697 12616 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:17.115398 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.114938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41220)
0811 06:56:17.115142 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:17.115233 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:56:17.116647 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.116235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41232)
0811 06:56:17.116437 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:17.116517 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 06:56:17.157684 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12388
I20260811 06:56:17.180639 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11763
I20260811 06:56:17.181850 11860 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:17.291328 12710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.290544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34879 (local address 127.0.0.1:41242)
0811 06:56:17.291044 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:17.291184 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:34879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209}
I20260811 06:56:17.291322 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11763
2026-08-11T06:56:17Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4719 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T06:56:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:17Z Disabled control of system clock
I20260811 06:56:17.377009 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43905
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:43905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:56:17.707408 12722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:17.707667 12722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:17.707749 12722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:17.714687 12722 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:17.714793 12722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:17.714831 12722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:17.714880 12722 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:17.714916 12722 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:17.723964 12722 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:43905
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:17.725366 12722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:17.727180 12722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:17.739114 12730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:17.739080 12731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:17.739046 12734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:17.740381 12722 server_base.cc:1034] running on GCE node
I20260811 06:56:17.741226 12722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:17.742578 12722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:17.743798 12722 hybrid_clock.cc:648] HybridClock initialized: now 1786431377743737 us; error 56 us; skew 500 ppm
I20260811 06:56:17.744227 12722 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:17.746532 12722 webserver.cc:460] Webserver started at http://127.4.40.62:45943/ using document root <none> and password file <none>
I20260811 06:56:17.747217 12722 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:17.747344 12722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:17.747709 12722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:17.750631 12722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "0ea6564e56b44c67bd693d4fd5408022"
format_stamp: "Formatted at 2026-08-11 06:56:17 on dist-test-slave-7vjv"
server_key: "ad9eac0d8c3286701714f203c9f96e6e"
server_key_iv: "a96c213df74a0372ebb24d21b47035f3"
server_key_version: "encryptionkey@0"
I20260811 06:56:17.751428 12722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "0ea6564e56b44c67bd693d4fd5408022"
format_stamp: "Formatted at 2026-08-11 06:56:17 on dist-test-slave-7vjv"
server_key: "ad9eac0d8c3286701714f203c9f96e6e"
server_key_iv: "a96c213df74a0372ebb24d21b47035f3"
server_key_version: "encryptionkey@0"
I20260811 06:56:17.756574 12722 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:56:17.759891 12743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:17.761190 12722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:17.761364 12722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "0ea6564e56b44c67bd693d4fd5408022"
format_stamp: "Formatted at 2026-08-11 06:56:17 on dist-test-slave-7vjv"
server_key: "ad9eac0d8c3286701714f203c9f96e6e"
server_key_iv: "a96c213df74a0372ebb24d21b47035f3"
server_key_version: "encryptionkey@0"
I20260811 06:56:17.761574 12722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:17.767274 12722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:17.768038 12722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:17.768371 12722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:17.786356 12722 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:43905
I20260811 06:56:17.786396 12833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:43905 every 8 connection(s)
I20260811 06:56:17.787663 12722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:17.791208 12834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:17.797225 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12722
I20260811 06:56:17.797437 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:17.798096 4256 external_mini_cluster.cc:1428] Setting key 87b48627a618ac5a3d3ed829e3d34444
I20260811 06:56:17.802213 12834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022: Bootstrap starting.
I20260811 06:56:17.806249 12834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:17.807626 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:17.800072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.4.40.62:43905)
0811 06:56:17.801176 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:17.801190 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:17.801244 (+ 54us) server_negotiation.cc:407] Connection header received
0811 06:56:17.803534 (+ 2290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:17.803561 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:17.804112 (+ 551us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:17.804598 (+ 486us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:17.805575 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:17.805582 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:17.805598 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:17.805615 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:17.805689 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:17.805709 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:17.805897 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:17.806037 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 06:56:17.806304 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":300,"threads_started":1}
I20260811 06:56:17.807726 12834 log.cc:824] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:17.811594 12834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022: No bootstrap required, opened a new log
I20260811 06:56:17.819656 12834 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea6564e56b44c67bd693d4fd5408022" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43905 } }
I20260811 06:56:17.820225 12834 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:17.820376 12834 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ea6564e56b44c67bd693d4fd5408022, State: Initialized, Role: FOLLOWER
I20260811 06:56:17.821036 12834 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea6564e56b44c67bd693d4fd5408022" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43905 } }
I20260811 06:56:17.821264 12834 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:17.821375 12834 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:17.821501 12834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:17.823683 12834 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea6564e56b44c67bd693d4fd5408022" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43905 } }
I20260811 06:56:17.824270 12834 leader_election.cc:302] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ea6564e56b44c67bd693d4fd5408022; no voters:
I20260811 06:56:17.824810 12834 leader_election.cc:288] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:17.824931 12844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:17.825783 12844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [term 1 LEADER]: Becoming Leader. State: Replica: 0ea6564e56b44c67bd693d4fd5408022, State: Running, Role: LEADER
I20260811 06:56:17.826372 12844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea6564e56b44c67bd693d4fd5408022" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43905 } }
I20260811 06:56:17.827355 12834 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:17.829591 12851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ea6564e56b44c67bd693d4fd5408022. Latest consensus state: current_term: 1 leader_uuid: "0ea6564e56b44c67bd693d4fd5408022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea6564e56b44c67bd693d4fd5408022" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43905 } } }
I20260811 06:56:17.830009 12851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:17.829751 12850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ea6564e56b44c67bd693d4fd5408022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ea6564e56b44c67bd693d4fd5408022" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43905 } } }
I20260811 06:56:17.830233 12850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:17.836510 12854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:17.841373 12854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:17.849440 12854 catalog_manager.cc:1347] Generated new cluster ID: 5b3f72098ad94b6a9283404b785ab2fa
I20260811 06:56:17.849552 12854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:17.869683 12854 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:17.870244 12854 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:17.891407 12854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022: Generated new TSK 0
I20260811 06:56:17.892373 12854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:56:17.905313 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43905
--builtin_ntp_servers=127.4.40.20:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 06:56:18.238057 12880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:18.238319 12880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:18.238476 12880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:18.245602 12880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:18.245698 12880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:18.245875 12880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:18.254515 12880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:18.255816 12880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:18.257596 12880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:18.270773 12893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:18.270818 12896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:18.270862 12894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:18.271919 12880 server_base.cc:1034] running on GCE node
I20260811 06:56:18.272673 12880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:18.273586 12880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:18.274807 12880 hybrid_clock.cc:648] HybridClock initialized: now 1786431378274762 us; error 51 us; skew 500 ppm
I20260811 06:56:18.275126 12880 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:18.277473 12880 webserver.cc:460] Webserver started at http://127.4.40.1:44087/ using document root <none> and password file <none>
I20260811 06:56:18.278138 12880 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:18.278283 12880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:18.278597 12880 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:18.281526 12880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "7d2660f5c53f4540a367d13651b8a9ba"
format_stamp: "Formatted at 2026-08-11 06:56:18 on dist-test-slave-7vjv"
server_key: "36c10b6fc85808e4bce36fb84336624b"
server_key_iv: "d1f2dea6f13a578ac6b8099379d65894"
server_key_version: "encryptionkey@0"
I20260811 06:56:18.282342 12880 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "7d2660f5c53f4540a367d13651b8a9ba"
format_stamp: "Formatted at 2026-08-11 06:56:18 on dist-test-slave-7vjv"
server_key: "36c10b6fc85808e4bce36fb84336624b"
server_key_iv: "d1f2dea6f13a578ac6b8099379d65894"
server_key_version: "encryptionkey@0"
I20260811 06:56:18.287346 12880 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:56:18.290699 12910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:18.291970 12880 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:56:18.292155 12880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "7d2660f5c53f4540a367d13651b8a9ba"
format_stamp: "Formatted at 2026-08-11 06:56:18 on dist-test-slave-7vjv"
server_key: "36c10b6fc85808e4bce36fb84336624b"
server_key_iv: "d1f2dea6f13a578ac6b8099379d65894"
server_key_version: "encryptionkey@0"
I20260811 06:56:18.292388 12880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:18.297824 12880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:18.298521 12880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:18.298746 12880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:18.299603 12880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:18.301036 12880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:18.301126 12880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:18.301208 12880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:18.301265 12880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:18.335371 12880 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:42599
I20260811 06:56:18.335461 13117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:42599 every 8 connection(s)
I20260811 06:56:18.337354 12880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:18.341470 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12880
I20260811 06:56:18.341688 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:18.342581 4256 external_mini_cluster.cc:1428] Setting key 1ceb2145e27222ce96c94592691c4861
I20260811 06:56:18.346068 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43905
--builtin_ntp_servers=127.4.40.20:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:56:18.347240 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:18.340292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:38545 (local address 127.4.40.62:43905)
0811 06:56:18.340956 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:18.340963 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:18.342780 (+ 1817us) server_negotiation.cc:407] Connection header received
0811 06:56:18.344820 (+ 2040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:18.344825 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:18.344892 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:18.345016 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:18.346443 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:18.346449 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:18.346453 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:18.346472 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:18.346530 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:18.346542 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:18.346664 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:18.346871 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 06:56:18.346942 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":332,"threads_started":1}
I20260811 06:56:18.348297 13129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:18.340977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43905 (local address 127.4.40.1:38545)
0811 06:56:18.342613 (+ 1636us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:18.342654 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:18.344524 (+ 1870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:18.345279 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:18.345287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:18.345373 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:18.346092 (+ 719us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:18.346095 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:18.346150 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:18.346277 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:18.346638 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:18.346651 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:18.346659 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:56:18.346961 (+ 302us) client_negotiation.cc:240] Negotiation successful
0811 06:56:18.347252 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":597,"threads_started":1}
I20260811 06:56:18.349798 13120 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43905
I20260811 06:56:18.350152 13120 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:18.350850 13120 heartbeater.cc:511] Master 127.4.40.62:43905 requested a full tablet report, sending...
I20260811 06:56:18.353055 12779 ts_manager.cc:194] Registered new tserver with Master: 7d2660f5c53f4540a367d13651b8a9ba (127.4.40.1:42599)
W20260811 06:56:18.707347 13130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:18.707656 13130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:18.707808 13130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:18.715180 13130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:18.715310 13130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:18.715484 13130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:18.724422 13130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:18.725718 13130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:18.727622 13130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:18.740721 13144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:18.740821 13146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:18.741060 13142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:18.741832 13130 server_base.cc:1034] running on GCE node
I20260811 06:56:18.742484 13130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:18.743379 13130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:18.744612 13130 hybrid_clock.cc:648] HybridClock initialized: now 1786431378744557 us; error 57 us; skew 500 ppm
I20260811 06:56:18.744948 13130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:18.747123 13130 webserver.cc:460] Webserver started at http://127.4.40.2:46883/ using document root <none> and password file <none>
I20260811 06:56:18.747818 13130 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:18.747952 13130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:18.748342 13130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:18.751296 13130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "670c1190b4b44495b32c8615c8bbc225"
format_stamp: "Formatted at 2026-08-11 06:56:18 on dist-test-slave-7vjv"
server_key: "3658e5a2a1bc21e603f882d4f39fa5e1"
server_key_iv: "8a08f868e06adafd19650dcb7807e4d6"
server_key_version: "encryptionkey@0"
I20260811 06:56:18.752138 13130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "670c1190b4b44495b32c8615c8bbc225"
format_stamp: "Formatted at 2026-08-11 06:56:18 on dist-test-slave-7vjv"
server_key: "3658e5a2a1bc21e603f882d4f39fa5e1"
server_key_iv: "8a08f868e06adafd19650dcb7807e4d6"
server_key_version: "encryptionkey@0"
I20260811 06:56:18.757248 13130 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:56:18.760764 13155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:18.762176 13130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:56:18.762349 13130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "670c1190b4b44495b32c8615c8bbc225"
format_stamp: "Formatted at 2026-08-11 06:56:18 on dist-test-slave-7vjv"
server_key: "3658e5a2a1bc21e603f882d4f39fa5e1"
server_key_iv: "8a08f868e06adafd19650dcb7807e4d6"
server_key_version: "encryptionkey@0"
I20260811 06:56:18.762588 13130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:18.768471 13130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:18.769268 13130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:18.769555 13130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:18.770485 13130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:18.771907 13130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:18.772049 13130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:18.772197 13130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:18.772289 13130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:18.810343 13130 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:37599
I20260811 06:56:18.810402 13357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:37599 every 8 connection(s)
I20260811 06:56:18.811676 13130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:18.812646 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13130
I20260811 06:56:18.812812 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:18.813359 4256 external_mini_cluster.cc:1428] Setting key 1c72cf888b960bcc29d2a8fed9b58fcb
I20260811 06:56:18.816650 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43905
--builtin_ntp_servers=127.4.40.20:36569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 06:56:18.821605 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:18.814425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:53619 (local address 127.4.40.62:43905)
0811 06:56:18.814640 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:18.814645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:18.816262 (+ 1617us) server_negotiation.cc:407] Connection header received
0811 06:56:18.819194 (+ 2932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:18.819199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:18.819251 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:18.819339 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:18.820897 (+ 1558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:18.820901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:18.820904 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:18.820921 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:18.820953 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:18.820976 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:18.821075 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:18.821352 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:56:18.821402 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:56:18.822916 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:18.814959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43905 (local address 127.4.40.2:53619)
0811 06:56:18.816068 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:18.816148 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:18.818896 (+ 2748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:18.819618 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:18.819631 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:18.819745 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:18.820509 (+ 764us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:18.820514 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:18.820587 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:18.820707 (+ 120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:18.821118 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:18.821134 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:18.821146 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 06:56:18.821459 (+ 313us) client_negotiation.cc:240] Negotiation successful
0811 06:56:18.821709 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":332,"threads_started":1}
I20260811 06:56:18.824184 13359 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43905
I20260811 06:56:18.824484 13359 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:18.825052 13359 heartbeater.cc:511] Master 127.4.40.62:43905 requested a full tablet report, sending...
I20260811 06:56:18.826380 12779 ts_manager.cc:194] Registered new tserver with Master: 670c1190b4b44495b32c8615c8bbc225 (127.4.40.2:37599)
W20260811 06:56:19.151233 13363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:19.151484 13363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:19.151638 13363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:19.158641 13363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:19.158730 13363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:19.158885 13363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:19.167829 13363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:36569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43905
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:19.169137 13363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:19.170895 13363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:19.183949 13369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:19.183979 13371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:19.184168 13373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:19.184921 13363 server_base.cc:1034] running on GCE node
I20260811 06:56:19.185586 13363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:19.186540 13363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:19.187965 13363 hybrid_clock.cc:648] HybridClock initialized: now 1786431379187916 us; error 51 us; skew 500 ppm
I20260811 06:56:19.188375 13363 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:19.190548 13363 webserver.cc:460] Webserver started at http://127.4.40.3:46837/ using document root <none> and password file <none>
I20260811 06:56:19.191231 13363 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:19.191361 13363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:19.191685 13363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:19.194628 13363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe"
format_stamp: "Formatted at 2026-08-11 06:56:19 on dist-test-slave-7vjv"
server_key: "0cf66381edf8460a389cf6d05eebb438"
server_key_iv: "b7257c74c6879c655b405e203e711856"
server_key_version: "encryptionkey@0"
I20260811 06:56:19.195430 13363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe"
format_stamp: "Formatted at 2026-08-11 06:56:19 on dist-test-slave-7vjv"
server_key: "0cf66381edf8460a389cf6d05eebb438"
server_key_iv: "b7257c74c6879c655b405e203e711856"
server_key_version: "encryptionkey@0"
I20260811 06:56:19.200788 13363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:56:19.204276 13389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:19.205549 13363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:56:19.205726 13363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe"
format_stamp: "Formatted at 2026-08-11 06:56:19 on dist-test-slave-7vjv"
server_key: "0cf66381edf8460a389cf6d05eebb438"
server_key_iv: "b7257c74c6879c655b405e203e711856"
server_key_version: "encryptionkey@0"
I20260811 06:56:19.205962 13363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:19.211531 13363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:19.212431 13363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:19.212699 13363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:19.213595 13363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:19.214974 13363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:19.215106 13363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:19.215246 13363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:19.215335 13363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:19.248636 13363 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:45821
I20260811 06:56:19.248690 13607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:45821 every 8 connection(s)
I20260811 06:56:19.249972 13363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:19.253201 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13363
I20260811 06:56:19.253419 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:19.254010 4256 external_mini_cluster.cc:1428] Setting key 26dc49abc7d26c2012b6dcfa74c19e12
I20260811 06:56:19.258569 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.252530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:59125 (local address 127.4.40.62:43905)
0811 06:56:19.252705 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.252709 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.254262 (+ 1553us) server_negotiation.cc:407] Connection header received
0811 06:56:19.256526 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.256532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.256614 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.256725 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.258002 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.258006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.258009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.258025 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.258057 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.258068 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.258168 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.258314 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.258362 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 06:56:19.259444 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.252975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43905 (local address 127.4.40.3:59125)
0811 06:56:19.254077 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.254126 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:19.256284 (+ 2158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:19.256967 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:19.256975 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:19.257110 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:19.257735 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:19.257738 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:19.257772 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:19.257849 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:19.258150 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:19.258157 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:19.258163 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:56:19.258364 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 06:56:19.258538 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":331,"threads_started":1}
I20260811 06:56:19.260761 13608 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43905
I20260811 06:56:19.261005 13608 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:19.261570 13608 heartbeater.cc:511] Master 127.4.40.62:43905 requested a full tablet report, sending...
I20260811 06:56:19.262782 12779 ts_manager.cc:194] Registered new tserver with Master: fd9e1a7bb8b04d8da7efcf0c98ef8afe (127.4.40.3:45821)
I20260811 06:56:19.269025 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:56:19.275663 13127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.273437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.4.40.62:43905)
0811 06:56:19.273592 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.273596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.274205 (+ 609us) server_negotiation.cc:407] Connection header received
0811 06:56:19.274334 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.274337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.274385 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.274462 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.275238 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.275241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.275245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.275259 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.275289 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.275301 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.275400 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.275432 (+ 32us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.275476 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:56:19.279302 12779 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:56:19.286020 12779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:19.320119 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.297472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:45821 (local address 127.0.0.1:52090)
0811 06:56:19.313010 (+ 15538us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.313068 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:19.313239 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:19.316890 (+ 3651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:19.316897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:19.316999 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:19.317837 (+ 838us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:19.317840 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:19.317881 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:19.317967 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:19.319323 (+ 1356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:19.319335 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:19.319345 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:56:19.319519 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 06:56:19.319630 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15173,"thread_start_us":18313,"threads_started":1}
I20260811 06:56:19.320189 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.298334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40234 (local address 127.4.40.1:42599)
0811 06:56:19.312188 (+ 13854us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.312202 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.316242 (+ 4040us) server_negotiation.cc:407] Connection header received
0811 06:56:19.316405 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.316414 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.316661 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.316819 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.319192 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.319199 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.319209 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.319239 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.319299 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.319321 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.319442 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.319722 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.319831 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13578,"thread_start_us":13397,"threads_started":1}
I20260811 06:56:19.321321 13630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.297652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:40234)
0811 06:56:19.316082 (+ 18430us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.316143 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:19.316252 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:19.318189 (+ 1937us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:19.318194 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:19.318216 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:19.318960 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:19.318963 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:19.318979 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:19.319022 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:19.319538 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:19.319542 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:19.319546 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 06:56:19.320764 (+ 1218us) client_negotiation.cc:240] Negotiation successful
0811 06:56:19.320821 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18295,"thread_start_us":14790,"threads_started":1}
I20260811 06:56:19.322227 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.297686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52090 (local address 127.4.40.3:45821)
0811 06:56:19.314173 (+ 16487us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.314185 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.314232 (+ 47us) server_negotiation.cc:407] Connection header received
0811 06:56:19.316443 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.316451 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.316661 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.317486 (+ 825us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.319074 (+ 1588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.319086 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.319100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.319121 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.319179 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.319200 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.321821 (+ 2621us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.321891 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.321962 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16265,"thread_start_us":16032,"threads_started":1}
I20260811 06:56:19.324751 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.314103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:37599 (local address 127.0.0.1:44744)
0811 06:56:19.320359 (+ 6256us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.320377 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:19.321563 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:19.322496 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:19.322501 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:19.322536 (+ 35us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:19.323052 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:19.323055 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:19.323071 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:19.323116 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:19.324321 (+ 1205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:19.324326 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:19.324331 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:56:19.324438 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 06:56:19.324491 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6162,"thread_start_us":7051,"threads_started":1}
I20260811 06:56:19.324847 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.313875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.4.40.2:37599)
0811 06:56:19.321850 (+ 7975us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.321860 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.321890 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:56:19.321963 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.321972 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.322189 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.322341 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.324056 (+ 1715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.324064 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.324076 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.324119 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.324173 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.324192 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.324325 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.324468 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.324561 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7775,"thread_start_us":7647,"threads_started":1}
I20260811 06:56:19.325675 12997 tablet_service.cc:1467] Processing CreateTablet for tablet 07cb127b22d642a58ef71eb2d29dcd8f (DEFAULT_TABLE table=test-table [id=b8fce24cb152489d88cea8c82ed60a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:19.325675 13477 tablet_service.cc:1467] Processing CreateTablet for tablet 07cb127b22d642a58ef71eb2d29dcd8f (DEFAULT_TABLE table=test-table [id=b8fce24cb152489d88cea8c82ed60a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:19.326889 13477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07cb127b22d642a58ef71eb2d29dcd8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:19.327001 12997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07cb127b22d642a58ef71eb2d29dcd8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:19.329299 13239 tablet_service.cc:1467] Processing CreateTablet for tablet 07cb127b22d642a58ef71eb2d29dcd8f (DEFAULT_TABLE table=test-table [id=b8fce24cb152489d88cea8c82ed60a0e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:19.330663 13239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 07cb127b22d642a58ef71eb2d29dcd8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:19.348810 13646 tablet_bootstrap.cc:492] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Bootstrap starting.
I20260811 06:56:19.352407 13646 tablet_bootstrap.cc:654] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:19.353416 13646 log.cc:824] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:19.346910 13639 tablet_bootstrap.cc:492] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: Bootstrap starting.
I20260811 06:56:19.356137 13120 heartbeater.cc:503] Master 127.4.40.62:43905 was elected leader, sending a full tablet report...
I20260811 06:56:19.356822 13643 tablet_bootstrap.cc:492] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Bootstrap starting.
I20260811 06:56:19.358466 13639 tablet_bootstrap.cc:654] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:19.358687 13646 tablet_bootstrap.cc:492] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: No bootstrap required, opened a new log
I20260811 06:56:19.359001 13646 ts_tablet_manager.cc:1397] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:56:19.359855 13639 log.cc:824] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:19.360436 13643 tablet_bootstrap.cc:654] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:19.361930 13643 log.cc:824] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:19.364794 13639 tablet_bootstrap.cc:492] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: No bootstrap required, opened a new log
I20260811 06:56:19.365069 13639 ts_tablet_manager.cc:1397] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.000s
I20260811 06:56:19.365208 13643 tablet_bootstrap.cc:492] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: No bootstrap required, opened a new log
I20260811 06:56:19.365473 13643 ts_tablet_manager.cc:1397] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.002s
I20260811 06:56:19.369236 13646 raft_consensus.cc:348] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.369688 13646 raft_consensus.cc:374] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:19.369848 13646 raft_consensus.cc:729] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 670c1190b4b44495b32c8615c8bbc225, State: Initialized, Role: FOLLOWER
I20260811 06:56:19.370612 13646 consensus_queue.cc:260] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.372773 13359 heartbeater.cc:503] Master 127.4.40.62:43905 was elected leader, sending a full tablet report...
I20260811 06:56:19.373035 13639 raft_consensus.cc:348] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.373381 13639 raft_consensus.cc:374] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:19.373497 13639 raft_consensus.cc:729] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2660f5c53f4540a367d13651b8a9ba, State: Initialized, Role: FOLLOWER
I20260811 06:56:19.374078 13639 consensus_queue.cc:260] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.374754 13646 ts_tablet_manager.cc:1428] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Time spent starting tablet: real 0.016s user 0.006s sys 0.007s
I20260811 06:56:19.376348 13639 ts_tablet_manager.cc:1428] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 06:56:19.376061 13643 raft_consensus.cc:348] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.376523 13643 raft_consensus.cc:374] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:19.376700 13643 raft_consensus.cc:729] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9e1a7bb8b04d8da7efcf0c98ef8afe, State: Initialized, Role: FOLLOWER
I20260811 06:56:19.377300 13643 consensus_queue.cc:260] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.378919 13608 heartbeater.cc:503] Master 127.4.40.62:43905 was elected leader, sending a full tablet report...
I20260811 06:56:19.379300 13643 ts_tablet_manager.cc:1428] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Time spent starting tablet: real 0.014s user 0.006s sys 0.007s
I20260811 06:56:19.464927 13658 raft_consensus.cc:482] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:19.465276 13658 raft_consensus.cc:504] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.467198 13658 leader_election.cc:288] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d2660f5c53f4540a367d13651b8a9ba (127.4.40.1:42599), 670c1190b4b44495b32c8615c8bbc225 (127.4.40.2:37599)
I20260811 06:56:19.470043 13611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.467358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.4.40.3:45137)
0811 06:56:19.467512 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.467533 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:19.467632 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:19.468271 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:19.468275 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:19.468298 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:19.469100 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:19.469104 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:19.469128 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:19.469190 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:19.469581 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:19.469586 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:19.469591 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:56:19.469717 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 06:56:19.469770 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 06:56:19.470070 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.467559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:45137 (local address 127.4.40.1:42599)
0811 06:56:19.467733 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.467741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.467762 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:56:19.467822 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.467826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.467889 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.468018 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.469344 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.469349 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.469352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.469388 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.469428 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.469446 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.469573 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.469718 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.469771 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:56:19.470644 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.467888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:50143 (local address 127.4.40.2:37599)
0811 06:56:19.468189 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.468195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.468541 (+ 346us) server_negotiation.cc:407] Connection header received
0811 06:56:19.468763 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.468770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.468834 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.468940 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.469927 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.469933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.469938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.469958 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.469990 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.470004 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.470151 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.470298 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.470365 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 06:56:19.471419 13660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.467688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:37599 (local address 127.4.40.3:50143)
0811 06:56:19.468414 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.468431 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:19.468601 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:19.468987 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:19.468992 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:19.469020 (+ 28us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:19.469716 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:56:19.469719 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:19.469736 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:56:19.469777 (+ 41us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:19.470119 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:19.470123 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:19.470128 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:56:19.470872 (+ 744us) client_negotiation.cc:240] Negotiation successful
0811 06:56:19.470940 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":335,"threads_started":1}
I20260811 06:56:19.471496 13034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cb127b22d642a58ef71eb2d29dcd8f" candidate_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2660f5c53f4540a367d13651b8a9ba" is_pre_election: true
I20260811 06:56:19.472154 13034 raft_consensus.cc:2454] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd9e1a7bb8b04d8da7efcf0c98ef8afe in term 0.
I20260811 06:56:19.472536 13278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cb127b22d642a58ef71eb2d29dcd8f" candidate_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670c1190b4b44495b32c8615c8bbc225" is_pre_election: true
I20260811 06:56:19.472805 13398 leader_election.cc:302] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d2660f5c53f4540a367d13651b8a9ba, fd9e1a7bb8b04d8da7efcf0c98ef8afe; no voters:
I20260811 06:56:19.473129 13278 raft_consensus.cc:2454] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fd9e1a7bb8b04d8da7efcf0c98ef8afe in term 0.
I20260811 06:56:19.473176 13658 raft_consensus.cc:2781] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:19.473310 13658 raft_consensus.cc:482] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:19.473430 13658 raft_consensus.cc:3037] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:19.475672 13658 raft_consensus.cc:504] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.476272 13658 leader_election.cc:288] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [CANDIDATE]: Term 1 election: Requested vote from peers 7d2660f5c53f4540a367d13651b8a9ba (127.4.40.1:42599), 670c1190b4b44495b32c8615c8bbc225 (127.4.40.2:37599)
I20260811 06:56:19.476670 13034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cb127b22d642a58ef71eb2d29dcd8f" candidate_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2660f5c53f4540a367d13651b8a9ba"
I20260811 06:56:19.476765 13278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "07cb127b22d642a58ef71eb2d29dcd8f" candidate_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "670c1190b4b44495b32c8615c8bbc225"
I20260811 06:56:19.476889 13034 raft_consensus.cc:3037] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:19.476960 13278 raft_consensus.cc:3037] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:19.480275 13278 raft_consensus.cc:2454] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd9e1a7bb8b04d8da7efcf0c98ef8afe in term 1.
I20260811 06:56:19.480275 13034 raft_consensus.cc:2454] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fd9e1a7bb8b04d8da7efcf0c98ef8afe in term 1.
I20260811 06:56:19.480651 13398 leader_election.cc:302] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d2660f5c53f4540a367d13651b8a9ba, fd9e1a7bb8b04d8da7efcf0c98ef8afe; no voters:
I20260811 06:56:19.480932 13658 raft_consensus.cc:2781] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:19.481384 13658 raft_consensus.cc:686] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 1 LEADER]: Becoming Leader. State: Replica: fd9e1a7bb8b04d8da7efcf0c98ef8afe, State: Running, Role: LEADER
I20260811 06:56:19.481874 13658 consensus_queue.cc:237] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } }
I20260811 06:56:19.486320 12775 catalog_manager.cc:5495] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe reported cstate change: term changed from 0 to 1, leader changed from <none> to fd9e1a7bb8b04d8da7efcf0c98ef8afe (127.4.40.3). New cstate: current_term: 1 leader_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd9e1a7bb8b04d8da7efcf0c98ef8afe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 } health_report { overall_health: UNKNOWN } } }
I20260811 06:56:19.500082 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.498235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.4.40.3:45821)
0811 06:56:19.498387 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.498391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.498492 (+ 101us) server_negotiation.cc:407] Connection header received
0811 06:56:19.498677 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.498680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.498753 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.498829 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.499621 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.499625 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.499628 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.499643 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.499672 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.499683 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.499780 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.499841 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.499885 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 06:56:19.501632 13609 tablet.cc:2369] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:19.508361 13034 raft_consensus.cc:1261] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 1 FOLLOWER]: Refusing update from remote peer fd9e1a7bb8b04d8da7efcf0c98ef8afe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:19.508360 13278 raft_consensus.cc:1261] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 1 FOLLOWER]: Refusing update from remote peer fd9e1a7bb8b04d8da7efcf0c98ef8afe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:19.509384 13661 consensus_queue.cc:1035] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [LEADER]: Connected to new peer: Peer: permanent_uuid: "670c1190b4b44495b32c8615c8bbc225" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:19.509809 13658 consensus_queue.cc:1035] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d2660f5c53f4540a367d13651b8a9ba" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42599 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:19.521152 13667 mvcc.cc:205] Tried to move back new op lower bound from 7317222930455228416 to 7317222930361434112. Current Snapshot: MvccSnapshot[applied={T|T < 7317222930455228416}]
I20260811 06:56:19.523126 13671 mvcc.cc:205] Tried to move back new op lower bound from 7317222930455228416 to 7317222930361434112. Current Snapshot: MvccSnapshot[applied={T|T < 7317222930455228416}]
I20260811 06:56:19.538530 12779 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:19.538828 12779 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:19.542590 12779 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ea6564e56b44c67bd693d4fd5408022: Sending DeleteTablet for 3 replicas of tablet 07cb127b22d642a58ef71eb2d29dcd8f
I20260811 06:56:19.543972 13477 tablet_service.cc:1514] Processing DeleteTablet for tablet 07cb127b22d642a58ef71eb2d29dcd8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:19 UTC) from {username='slave'} at 127.0.0.1:52090
I20260811 06:56:19.543970 12997 tablet_service.cc:1514] Processing DeleteTablet for tablet 07cb127b22d642a58ef71eb2d29dcd8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:19 UTC) from {username='slave'} at 127.0.0.1:40234
I20260811 06:56:19.545141 13239 tablet_service.cc:1514] Processing DeleteTablet for tablet 07cb127b22d642a58ef71eb2d29dcd8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:19 UTC) from {username='slave'} at 127.0.0.1:44744
I20260811 06:56:19.545379 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12880
I20260811 06:56:19.545495 13680 tablet_replica.cc:330] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: stopping tablet replica
I20260811 06:56:19.545984 13680 raft_consensus.cc:2229] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:19.546815 13680 raft_consensus.cc:2258] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:19.549449 13680 ts_tablet_manager.cc:1905] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:19.559265 13680 ts_tablet_manager.cc:1918] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:19.559864 13682 tablet_replica.cc:330] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: stopping tablet replica
I20260811 06:56:19.559954 13681 tablet_replica.cc:330] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba: stopping tablet replica
I20260811 06:56:19.560433 13682 raft_consensus.cc:2229] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:19.560439 13681 raft_consensus.cc:2229] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:19.560770 13635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.547037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48826 (local address 127.4.40.1:42599)
0811 06:56:19.547269 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.547279 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.557753 (+ 10474us) server_negotiation.cc:407] Connection header received
0811 06:56:19.558206 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.558211 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.558326 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.558404 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.559880 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.559886 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.559891 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.559913 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.559958 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.559985 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.560141 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.560367 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.560437 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:56:19.560983 13681 raft_consensus.cc:2258] T 07cb127b22d642a58ef71eb2d29dcd8f P 7d2660f5c53f4540a367d13651b8a9ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:19.561517 13680 log.cc:1196] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/07cb127b22d642a58ef71eb2d29dcd8f
I20260811 06:56:19.561818 13113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:19.562110 13680 ts_tablet_manager.cc:1939] T 07cb127b22d642a58ef71eb2d29dcd8f P fd9e1a7bb8b04d8da7efcf0c98ef8afe: Deleting consensus metadata
I20260811 06:56:19.564496 12745 catalog_manager.cc:4841] TS fd9e1a7bb8b04d8da7efcf0c98ef8afe (127.4.40.3:45821): tablet 07cb127b22d642a58ef71eb2d29dcd8f (table test-table [id=b8fce24cb152489d88cea8c82ed60a0e]) successfully deleted
I20260811 06:56:19.564615 13682 raft_consensus.cc:2258] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:19.566746 13682 ts_tablet_manager.cc:1905] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:19.571429 13682 ts_tablet_manager.cc:1918] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:19.571614 13682 log.cc:1196] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/07cb127b22d642a58ef71eb2d29dcd8f
I20260811 06:56:19.572073 13682 ts_tablet_manager.cc:1939] T 07cb127b22d642a58ef71eb2d29dcd8f P 670c1190b4b44495b32c8615c8bbc225: Deleting consensus metadata
I20260811 06:56:19.573500 12751 catalog_manager.cc:4841] TS 670c1190b4b44495b32c8615c8bbc225 (127.4.40.2:37599): tablet 07cb127b22d642a58ef71eb2d29dcd8f (table test-table [id=b8fce24cb152489d88cea8c82ed60a0e]) successfully deleted
I20260811 06:56:19.670163 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12880
W20260811 06:56:19.706866 12751 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:42599 (error 108)
I20260811 06:56:19.707216 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13130
I20260811 06:56:19.709264 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.708734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48836)
0811 06:56:19.708929 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.709073 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 06:56:19.709627 12751 catalog_manager.cc:4584] TS 7d2660f5c53f4540a367d13651b8a9ba (127.4.40.1:42599): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 07cb127b22d642a58ef71eb2d29dcd8f: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
I20260811 06:56:19.711145 13636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.708825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.4.40.2:37599)
0811 06:56:19.709018 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.709026 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.709047 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:56:19.709118 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.709123 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.709234 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.709357 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.710533 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.710537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.710541 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.710556 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.710586 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.710598 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.710691 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.710868 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.710930 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:56:19.711830 13351 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:19.763403 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.762789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48850)
0811 06:56:19.763083 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.763205 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 06:56:19.764720 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.764268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48856)
0811 06:56:19.764436 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.764537 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:56:19.809515 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13130
I20260811 06:56:19.825310 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.824892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48858)
0811 06:56:19.825083 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.825175 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:56:19.826542 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.826173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48870)
0811 06:56:19.826332 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.826415 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 06:56:19.843124 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13363
I20260811 06:56:19.846205 13632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.843914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44184 (local address 127.4.40.3:45821)
0811 06:56:19.844185 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:19.844192 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:19.844212 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:56:19.844426 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:19.844435 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:19.844485 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:19.844587 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:19.845515 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:19.845521 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:19.845526 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:19.845545 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:19.845582 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:19.845593 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:19.845741 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:19.845853 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 06:56:19.845910 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 06:56:19.846989 13599 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:19.907327 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.906819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48886)
0811 06:56:19.907065 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.907184 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 06:56:19.908428 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:19.907978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48892)
0811 06:56:19.908215 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:19.908296 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 06:56:19.965559 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13363
I20260811 06:56:19.997913 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12722
I20260811 06:56:19.999045 12827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:20.106086 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12722
I20260811 06:56:20.106154 13631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:20.105344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42599 (local address 127.0.0.1:48902)
0811 06:56:20.105720 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:20.105971 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42599: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192}
2026-08-11T06:56:20Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2816 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 06:56:20.161160 4256 mini_oidc.cc:152] Starting JWKS server
I20260811 06:56:20.164003 4256 webserver.cc:460] Webserver started at https://127.0.0.1:34865/ using document root <none> and password file <none>
I20260811 06:56:20.164387 4256 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:56:20.165045 4256 webserver.cc:460] Webserver started at http://127.0.0.1:39681/ using document root <none> and password file <none>
2026-08-11T06:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:20Z Disabled control of system clock
I20260811 06:56:20.200860 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:38787
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:46667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:38787
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34865/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/testchainca.cert with env {}
W20260811 06:56:20.535372 13714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:20.535688 13714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:20.535856 13714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:20.542929 13714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:20.543022 13714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:20.543058 13714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:20.543159 13714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:20.543222 13714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:56:20.543354 13714 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:56:20.543409 13714 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34865/jwks.json
I20260811 06:56:20.552917 13714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:38787
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:38787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34865/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:20.554155 13714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:20.555501 13714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:20.567239 13729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:20.567286 13727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:20.567319 13725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:20.568372 13714 server_base.cc:1034] running on GCE node
I20260811 06:56:20.569152 13714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:20.570487 13714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:20.571724 13714 hybrid_clock.cc:648] HybridClock initialized: now 1786431380571689 us; error 46 us; skew 500 ppm
I20260811 06:56:20.572141 13714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:20.574343 13714 webserver.cc:460] Webserver started at http://127.4.40.62:46763/ using document root <none> and password file <none>
I20260811 06:56:20.575018 13714 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:20.575156 13714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:20.575452 13714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:20.578418 13714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "0054ecbc08c24362bc5ed96453a2512f"
format_stamp: "Formatted at 2026-08-11 06:56:20 on dist-test-slave-7vjv"
server_key: "a217ee5f817edb1edad9a04a3c6f8eed"
server_key_iv: "4cdb1265e812d73a070d1349715fc3b8"
server_key_version: "encryptionkey@0"
I20260811 06:56:20.579218 13714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "0054ecbc08c24362bc5ed96453a2512f"
format_stamp: "Formatted at 2026-08-11 06:56:20 on dist-test-slave-7vjv"
server_key: "a217ee5f817edb1edad9a04a3c6f8eed"
server_key_iv: "4cdb1265e812d73a070d1349715fc3b8"
server_key_version: "encryptionkey@0"
I20260811 06:56:20.584649 13714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:56:20.588152 13738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:20.589453 13714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:56:20.589643 13714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "0054ecbc08c24362bc5ed96453a2512f"
format_stamp: "Formatted at 2026-08-11 06:56:20 on dist-test-slave-7vjv"
server_key: "a217ee5f817edb1edad9a04a3c6f8eed"
server_key_iv: "4cdb1265e812d73a070d1349715fc3b8"
server_key_version: "encryptionkey@0"
I20260811 06:56:20.589864 13714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:20.606881 13714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:20.607791 13714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:20.608081 13714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:20.626811 13714 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:38787
I20260811 06:56:20.626845 13834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:38787 every 8 connection(s)
I20260811 06:56:20.628170 13714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:20.631232 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13714
I20260811 06:56:20.631456 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:20.632365 4256 external_mini_cluster.cc:1428] Setting key 883dc475ab54f134f0f38a601645a4c7
I20260811 06:56:20.632494 13835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:20.646049 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:20.634242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.4.40.62:38787)
0811 06:56:20.635175 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:20.635193 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:20.635236 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:56:20.637151 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:20.637172 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:20.637600 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:20.637908 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:20.638542 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:20.639842 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:20.641682 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:20.642294 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:20.642422 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:20.643761 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:20.643769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:20.643781 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:20.644408 (+ 627us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:20.644494 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:20.644515 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:20.644600 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:20.644834 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 06:56:20.645153 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":343,"threads_started":1}
I20260811 06:56:20.646411 13835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f: Bootstrap starting.
I20260811 06:56:20.649940 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:56:20.650104 13835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:20.651278 13835 log.cc:824] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:20.654177 13835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f: No bootstrap required, opened a new log
I20260811 06:56:20.661523 13835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0054ecbc08c24362bc5ed96453a2512f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38787 } }
I20260811 06:56:20.661849 13835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:20.661926 13835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0054ecbc08c24362bc5ed96453a2512f, State: Initialized, Role: FOLLOWER
I20260811 06:56:20.662389 13835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0054ecbc08c24362bc5ed96453a2512f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38787 } }
I20260811 06:56:20.662590 13835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:20.662690 13835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:20.662812 13835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:20.664929 13835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0054ecbc08c24362bc5ed96453a2512f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38787 } }
I20260811 06:56:20.665413 13835 leader_election.cc:302] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0054ecbc08c24362bc5ed96453a2512f; no voters:
I20260811 06:56:20.666003 13835 leader_election.cc:288] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:20.666157 13844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:20.666903 13844 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [term 1 LEADER]: Becoming Leader. State: Replica: 0054ecbc08c24362bc5ed96453a2512f, State: Running, Role: LEADER
I20260811 06:56:20.667359 13844 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0054ecbc08c24362bc5ed96453a2512f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38787 } }
I20260811 06:56:20.668181 13835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:20.670132 13845 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0054ecbc08c24362bc5ed96453a2512f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0054ecbc08c24362bc5ed96453a2512f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38787 } } }
I20260811 06:56:20.670667 13845 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:20.670159 13847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0054ecbc08c24362bc5ed96453a2512f. Latest consensus state: current_term: 1 leader_uuid: "0054ecbc08c24362bc5ed96453a2512f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0054ecbc08c24362bc5ed96453a2512f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 38787 } } }
I20260811 06:56:20.672739 13847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:20.674189 13855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:20.683292 13855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:20.691735 13855 catalog_manager.cc:1347] Generated new cluster ID: b8cc99e51566482f929e38ee46ee094c
I20260811 06:56:20.691851 13855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:20.712412 13855 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:20.713886 13855 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:20.722642 13855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0054ecbc08c24362bc5ed96453a2512f: Generated new TSK 0
I20260811 06:56:20.723377 13855 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:56:20.771795 13885 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 06:56:20.772024 13885 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:56:20.773733 13841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:20.748227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.4.40.62:38787)
0811 06:56:20.748502 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:20.748511 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:20.749319 (+ 808us) server_negotiation.cc:407] Connection header received
0811 06:56:20.749849 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:20.749857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:20.749969 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:20.750158 (+ 189us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:56:20.750698 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:20.752040 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:20.753596 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:20.754453 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:20.754634 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:20.755259 (+ 625us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:56:20.772799 (+ 17540us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:56:20.773332 (+ 533us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 06:56:20.773895 13841 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 06:56:20.775224 13884 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:20.748030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:38787 (local address 127.0.0.1:33956)
0811 06:56:20.749199 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:20.749219 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:20.749384 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:20.750196 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:20.750201 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:20.750211 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:56:20.750533 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:20.750541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:20.752246 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:20.752251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:20.753424 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:20.753435 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:20.754626 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:20.754696 (+ 70us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:20.754880 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:20.754938 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:56:20.773186 (+ 18248us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 06:56:20.773449 (+ 263us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.40.62:38787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":401,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.40.62:38787: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34865/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 06:56:20.777453 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13714
I20260811 06:56:20.778735 13829 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:20.884004 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13714
2026-08-11T06:56:20Z chronyd exiting
I20260811 06:56:20.944787 4256 test_util.cc:175] -----------------------------------------------
I20260811 06:56:20.944986 4256 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786431333861527-4256-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (788 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 06:56:20.948874 4256 mini_oidc.cc:152] Starting JWKS server
I20260811 06:56:20.950661 4256 webserver.cc:460] Webserver started at https://127.0.0.1:35419/ using document root <none> and password file <none>
I20260811 06:56:20.950914 4256 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:56:20.951550 4256 webserver.cc:460] Webserver started at http://127.0.0.1:43809/ using document root <none> and password file <none>
2026-08-11T06:56:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:20Z Disabled control of system clock
I20260811 06:56:20.986301 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:39515
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:45155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:39515
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35419/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/testchainca.cert with env {}
W20260811 06:56:21.325878 13897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:21.326166 13897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:21.326362 13897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:21.333918 13897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:21.334018 13897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:21.334103 13897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:21.334205 13897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:21.334268 13897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:56:21.334376 13897 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:56:21.334431 13897 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35419/jwks.json
I20260811 06:56:21.344375 13897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:45155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:39515
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:39515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35419/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:21.345654 13897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:21.346956 13897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:21.358939 13913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:21.359180 13910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:21.359153 13916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:21.360381 13897 server_base.cc:1034] running on GCE node
I20260811 06:56:21.361148 13897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:21.362557 13897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:21.363794 13897 hybrid_clock.cc:648] HybridClock initialized: now 1786431381363763 us; error 47 us; skew 500 ppm
I20260811 06:56:21.364188 13897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:21.366318 13897 webserver.cc:460] Webserver started at http://127.4.40.62:35489/ using document root <none> and password file <none>
I20260811 06:56:21.366937 13897 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:21.367038 13897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:21.367302 13897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:21.370069 13897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "76709b539f0940c4a363ee26c8e502a5"
format_stamp: "Formatted at 2026-08-11 06:56:21 on dist-test-slave-7vjv"
server_key: "f516ef97a38efcdcb4282b77fdc23810"
server_key_iv: "1de9b9b22127fd032f317acf56676ca6"
server_key_version: "encryptionkey@0"
I20260811 06:56:21.370798 13897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "76709b539f0940c4a363ee26c8e502a5"
format_stamp: "Formatted at 2026-08-11 06:56:21 on dist-test-slave-7vjv"
server_key: "f516ef97a38efcdcb4282b77fdc23810"
server_key_iv: "1de9b9b22127fd032f317acf56676ca6"
server_key_version: "encryptionkey@0"
I20260811 06:56:21.375635 13897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 06:56:21.378979 13934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:21.380266 13897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:21.380446 13897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "76709b539f0940c4a363ee26c8e502a5"
format_stamp: "Formatted at 2026-08-11 06:56:21 on dist-test-slave-7vjv"
server_key: "f516ef97a38efcdcb4282b77fdc23810"
server_key_iv: "1de9b9b22127fd032f317acf56676ca6"
server_key_version: "encryptionkey@0"
I20260811 06:56:21.380651 13897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:21.400270 13897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:21.401048 13897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:21.401345 13897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:21.420703 13897 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:39515
I20260811 06:56:21.420724 14030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:39515 every 8 connection(s)
I20260811 06:56:21.422079 13897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:21.425748 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13897
I20260811 06:56:21.425958 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:21.426201 14032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:21.426887 4256 external_mini_cluster.cc:1428] Setting key df3cc5bd89a4d6f69e02015dd7e8123a
I20260811 06:56:21.439684 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5: Bootstrap starting.
I20260811 06:56:21.440954 14039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:21.428862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.4.40.62:39515)
0811 06:56:21.429931 (+ 1069us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:21.429942 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:21.429983 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:56:21.432386 (+ 2403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:21.432423 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:21.432845 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:21.433162 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:21.433732 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:21.434923 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:21.436795 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:21.437470 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:21.437604 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:21.438796 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:21.438802 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:21.438814 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:21.439288 (+ 474us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:21.439360 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:21.439378 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:21.439474 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:21.439752 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 06:56:21.440031 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":432,"threads_started":1}
I20260811 06:56:21.442899 14032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:21.444024 14032 log.cc:824] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:21.444422 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:56:21.446754 14032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5: No bootstrap required, opened a new log
I20260811 06:56:21.454208 14032 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76709b539f0940c4a363ee26c8e502a5" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39515 } }
I20260811 06:56:21.454532 14032 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:21.454597 14032 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76709b539f0940c4a363ee26c8e502a5, State: Initialized, Role: FOLLOWER
I20260811 06:56:21.455046 14032 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76709b539f0940c4a363ee26c8e502a5" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39515 } }
I20260811 06:56:21.455268 14032 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:21.455372 14032 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:21.455490 14032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:21.458204 14032 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76709b539f0940c4a363ee26c8e502a5" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39515 } }
I20260811 06:56:21.458626 14032 leader_election.cc:302] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76709b539f0940c4a363ee26c8e502a5; no voters:
I20260811 06:56:21.459091 14032 leader_election.cc:288] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:21.459288 14041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:21.460031 14041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [term 1 LEADER]: Becoming Leader. State: Replica: 76709b539f0940c4a363ee26c8e502a5, State: Running, Role: LEADER
I20260811 06:56:21.460583 14041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76709b539f0940c4a363ee26c8e502a5" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39515 } }
I20260811 06:56:21.461274 14032 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:21.463688 14042 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76709b539f0940c4a363ee26c8e502a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76709b539f0940c4a363ee26c8e502a5" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39515 } } }
I20260811 06:56:21.463660 14043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76709b539f0940c4a363ee26c8e502a5. Latest consensus state: current_term: 1 leader_uuid: "76709b539f0940c4a363ee26c8e502a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76709b539f0940c4a363ee26c8e502a5" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39515 } } }
I20260811 06:56:21.464139 14042 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:21.464155 14043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:21.472337 14049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:21.476622 14049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:21.485270 14049 catalog_manager.cc:1347] Generated new cluster ID: 5bb9d4bbbe3341f3a0e8703234d4c100
I20260811 06:56:21.485378 14049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:21.505537 14049 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:21.506865 14049 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:21.519480 14049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76709b539f0940c4a363ee26c8e502a5: Generated new TSK 0
I20260811 06:56:21.520421 14049 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 06:56:21.556036 14094 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 06:56:21.556187 14094 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:56:21.557240 14039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:21.537102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56648 (local address 127.4.40.62:39515)
0811 06:56:21.537280 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:21.537288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:21.537695 (+ 407us) server_negotiation.cc:407] Connection header received
0811 06:56:21.537924 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:21.537931 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:21.538010 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:21.538138 (+ 128us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:56:21.538686 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:21.539448 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:21.541549 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:21.541995 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:21.542090 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:21.542716 (+ 626us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:56:21.556692 (+ 13976us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:56:21.557008 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 06:56:21.557389 14039 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56648: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 06:56:21.557606 14092 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:21.536859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:39515 (local address 127.0.0.1:56648)
0811 06:56:21.537578 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:21.537600 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:21.537764 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:21.538188 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:21.538194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:21.538207 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:56:21.538499 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:21.538523 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:21.539608 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:21.539614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:21.541404 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:21.541421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:21.542122 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:21.542177 (+ 55us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:21.542401 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:21.542446 (+ 45us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:56:21.556912 (+ 14466us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 06:56:21.557192 (+ 280us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.40.62:39515: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35419/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":420,"threads_started":1}
I20260811 06:56:21.558938 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13897
I20260811 06:56:21.560060 14020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:21.654981 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13897
2026-08-11T06:56:21Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (773 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 06:56:21.722467 4256 mini_oidc.cc:152] Starting JWKS server
I20260811 06:56:21.724373 4256 webserver.cc:460] Webserver started at https://127.0.0.1:33523/ using document root <none> and password file <none>
I20260811 06:56:21.724581 4256 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 06:56:21.725149 4256 webserver.cc:460] Webserver started at http://127.0.0.1:36323/ using document root <none> and password file <none>
2026-08-11T06:56:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:21Z Disabled control of system clock
I20260811 06:56:21.763430 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:41097
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:35687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:41097
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33523/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:56:22.093392 14102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:22.093669 14102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:22.093853 14102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:22.100976 14102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:22.101092 14102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:22.101182 14102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:22.101290 14102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:22.101374 14102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:56:22.101464 14102 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:56:22.101528 14102 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33523/jwks.json
I20260811 06:56:22.110441 14102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:35687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:41097
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:41097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33523/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:22.111696 14102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:22.113497 14102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:22.125331 14110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:22.125459 14114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:22.125298 14118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:22.126511 14102 server_base.cc:1034] running on GCE node
I20260811 06:56:22.127218 14102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:22.128684 14102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:22.129897 14102 hybrid_clock.cc:648] HybridClock initialized: now 1786431382129839 us; error 49 us; skew 500 ppm
I20260811 06:56:22.130362 14102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:22.139199 14102 webserver.cc:460] Webserver started at http://127.4.40.62:37417/ using document root <none> and password file <none>
I20260811 06:56:22.139953 14102 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:22.140141 14102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:22.140492 14102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:22.143460 14102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "c00d3c40400d4613aacf94554b6dd6e2"
format_stamp: "Formatted at 2026-08-11 06:56:22 on dist-test-slave-7vjv"
server_key: "4ff2b6439464b4656456a92a15ca74a9"
server_key_iv: "6dcd37f9a333d047526493346da1ca3a"
server_key_version: "encryptionkey@0"
I20260811 06:56:22.144345 14102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "c00d3c40400d4613aacf94554b6dd6e2"
format_stamp: "Formatted at 2026-08-11 06:56:22 on dist-test-slave-7vjv"
server_key: "4ff2b6439464b4656456a92a15ca74a9"
server_key_iv: "6dcd37f9a333d047526493346da1ca3a"
server_key_version: "encryptionkey@0"
I20260811 06:56:22.149410 14102 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:56:22.152732 14125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:22.154100 14102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:56:22.154291 14102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "c00d3c40400d4613aacf94554b6dd6e2"
format_stamp: "Formatted at 2026-08-11 06:56:22 on dist-test-slave-7vjv"
server_key: "4ff2b6439464b4656456a92a15ca74a9"
server_key_iv: "6dcd37f9a333d047526493346da1ca3a"
server_key_version: "encryptionkey@0"
I20260811 06:56:22.154507 14102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:22.173563 14102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:22.174378 14102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:22.174661 14102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:22.193226 14102 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:41097
I20260811 06:56:22.193259 14216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:41097 every 8 connection(s)
I20260811 06:56:22.194667 14102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:22.198251 14220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:22.204716 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 14102
I20260811 06:56:22.204936 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:22.205636 4256 external_mini_cluster.cc:1428] Setting key 65d89c69be4e9e4f4e7c83003fe05e83
I20260811 06:56:22.208818 14220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2: Bootstrap starting.
I20260811 06:56:22.212229 14220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:22.213313 14220 log.cc:824] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:22.216488 14220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2: No bootstrap required, opened a new log
I20260811 06:56:22.219098 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:22.207433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49538 (local address 127.4.40.62:41097)
0811 06:56:22.208465 (+ 1032us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:22.208475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:22.208515 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:56:22.210305 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:22.210327 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:22.210753 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:22.211069 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:22.211704 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.213039 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.214674 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.215335 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.215470 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:22.216743 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:22.216750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:22.216763 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:22.217332 (+ 569us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:22.217405 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:22.217425 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:22.217517 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:22.217752 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 06:56:22.218093 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":356,"threads_started":1}
I20260811 06:56:22.223049 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:56:22.225637 14220 raft_consensus.cc:348] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00d3c40400d4613aacf94554b6dd6e2" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41097 } }
I20260811 06:56:22.226119 14220 raft_consensus.cc:374] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:22.226228 14220 raft_consensus.cc:729] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c00d3c40400d4613aacf94554b6dd6e2, State: Initialized, Role: FOLLOWER
I20260811 06:56:22.226899 14220 consensus_queue.cc:260] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00d3c40400d4613aacf94554b6dd6e2" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41097 } }
I20260811 06:56:22.227164 14220 raft_consensus.cc:388] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:22.227312 14220 raft_consensus.cc:482] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:22.227490 14220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:22.230089 14220 raft_consensus.cc:504] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00d3c40400d4613aacf94554b6dd6e2" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41097 } }
I20260811 06:56:22.230580 14220 leader_election.cc:302] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c00d3c40400d4613aacf94554b6dd6e2; no voters:
I20260811 06:56:22.231154 14220 leader_election.cc:288] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:22.231329 14233 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:22.232084 14233 raft_consensus.cc:686] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [term 1 LEADER]: Becoming Leader. State: Replica: c00d3c40400d4613aacf94554b6dd6e2, State: Running, Role: LEADER
I20260811 06:56:22.232637 14233 consensus_queue.cc:237] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00d3c40400d4613aacf94554b6dd6e2" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41097 } }
I20260811 06:56:22.233378 14220 sys_catalog.cc:564] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:22.235739 14235 sys_catalog.cc:455] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c00d3c40400d4613aacf94554b6dd6e2. Latest consensus state: current_term: 1 leader_uuid: "c00d3c40400d4613aacf94554b6dd6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00d3c40400d4613aacf94554b6dd6e2" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41097 } } }
I20260811 06:56:22.236227 14235 sys_catalog.cc:458] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:22.237592 14234 sys_catalog.cc:455] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c00d3c40400d4613aacf94554b6dd6e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c00d3c40400d4613aacf94554b6dd6e2" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41097 } } }
I20260811 06:56:22.237989 14234 sys_catalog.cc:458] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:22.238121 14241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:22.243398 14241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:22.252648 14241 catalog_manager.cc:1347] Generated new cluster ID: 66aedee5292e4f1f80fb21e9fe1c9360
I20260811 06:56:22.252769 14241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:22.266507 14241 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:22.268002 14241 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:22.275846 14241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c00d3c40400d4613aacf94554b6dd6e2: Generated new TSK 0
I20260811 06:56:22.276662 14241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:56:22.296694 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:22.288554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.4.40.62:41097)
0811 06:56:22.288770 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:22.288778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:22.289320 (+ 542us) server_negotiation.cc:407] Connection header received
0811 06:56:22.289528 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:22.289532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:22.289605 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:22.289729 (+ 124us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:56:22.290283 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.290992 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.293020 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.293466 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.293558 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:22.294150 (+ 592us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:56:22.296023 (+ 1873us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33523/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:56:22.296377 (+ 354us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33523/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 06:56:22.296835 14224 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49540: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33523/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 06:56:22.296947 14271 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:22.288540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:41097 (local address 127.0.0.1:49540)
0811 06:56:22.289210 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:22.289231 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:22.289381 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:22.289779 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:22.289784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:22.289798 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:56:22.290132 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:22.290146 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.291140 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.291146 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:22.292886 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:22.292900 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.293593 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.293671 (+ 78us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:22.293863 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:22.293919 (+ 56us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:56:22.296254 (+ 2335us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33523/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 06:56:22.296518 (+ 264us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.40.62:41097: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33523/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":347,"threads_started":1}
I20260811 06:56:22.298378 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 14102
I20260811 06:56:22.299482 14209 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 06:56:22.309041 14272 webserver.cc:533] Webserver: sslize failed:
E20260811 06:56:22.309155 14272 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 06:56:22.395077 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 14102
2026-08-11T06:56:22Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (751 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T06:56:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:22Z Disabled control of system clock
I20260811 06:56:22.510581 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:39379
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:37653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:39379
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/testchainca.cert with env {}
W20260811 06:56:22.840731 14280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:22.841073 14280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:22.841238 14280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:22.848330 14280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:22.848446 14280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:22.848527 14280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:22.848628 14280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:22.848708 14280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 06:56:22.848793 14280 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 06:56:22.848872 14280 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 06:56:22.858282 14280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:39379
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:39379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:22.859604 14280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:22.860934 14280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:22.872891 14288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:22.872848 14296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:22.872928 14293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:22.873842 14280 server_base.cc:1034] running on GCE node
I20260811 06:56:22.874604 14280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:22.875933 14280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:22.877163 14280 hybrid_clock.cc:648] HybridClock initialized: now 1786431382877126 us; error 47 us; skew 500 ppm
I20260811 06:56:22.877503 14280 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:22.879669 14280 webserver.cc:460] Webserver started at http://127.4.40.62:43779/ using document root <none> and password file <none>
I20260811 06:56:22.880430 14280 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:22.880574 14280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:22.880906 14280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:22.883775 14280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "46e03c5c9e7a476e99f51bdafa9949f3"
format_stamp: "Formatted at 2026-08-11 06:56:22 on dist-test-slave-7vjv"
server_key: "0117173f9a6f03e8e4a1f2cec2def142"
server_key_iv: "fc276a15b0775c422979d9ef14d16c23"
server_key_version: "encryptionkey@0"
I20260811 06:56:22.884624 14280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "46e03c5c9e7a476e99f51bdafa9949f3"
format_stamp: "Formatted at 2026-08-11 06:56:22 on dist-test-slave-7vjv"
server_key: "0117173f9a6f03e8e4a1f2cec2def142"
server_key_iv: "fc276a15b0775c422979d9ef14d16c23"
server_key_version: "encryptionkey@0"
I20260811 06:56:22.889532 14280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:56:22.893014 14307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:22.894356 14280 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:56:22.894564 14280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "46e03c5c9e7a476e99f51bdafa9949f3"
format_stamp: "Formatted at 2026-08-11 06:56:22 on dist-test-slave-7vjv"
server_key: "0117173f9a6f03e8e4a1f2cec2def142"
server_key_iv: "fc276a15b0775c422979d9ef14d16c23"
server_key_version: "encryptionkey@0"
I20260811 06:56:22.894786 14280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:22.915961 14280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:22.916837 14280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:22.917156 14280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:22.936511 14280 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:39379
I20260811 06:56:22.936544 14397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:39379 every 8 connection(s)
I20260811 06:56:22.937855 14280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:22.941593 14398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:22.943377 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 14280
I20260811 06:56:22.943576 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:22.944439 4256 external_mini_cluster.cc:1428] Setting key 2b3d3d15b04529c2ce8bd8e4e8f4db68
I20260811 06:56:22.954484 14398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3: Bootstrap starting.
I20260811 06:56:22.957333 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:22.946444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59788 (local address 127.4.40.62:39379)
0811 06:56:22.947404 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:22.947417 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:22.947463 (+ 46us) server_negotiation.cc:407] Connection header received
0811 06:56:22.949081 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:22.949105 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:22.949543 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:22.949892 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:22.950457 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.951723 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.953069 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:22.953687 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:22.953885 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:22.955123 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:22.955130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:22.955142 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:56:22.955647 (+ 505us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:22.955738 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:56:22.955768 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:22.955910 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:22.956176 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 06:56:22.956457 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":343,"threads_started":1}
I20260811 06:56:22.957749 14398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:22.958763 14398 log.cc:824] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:22.961141 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:56:22.961897 14398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3: No bootstrap required, opened a new log
I20260811 06:56:22.969925 14398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39379 } }
I20260811 06:56:22.970261 14398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:22.970402 14398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46e03c5c9e7a476e99f51bdafa9949f3, State: Initialized, Role: FOLLOWER
I20260811 06:56:22.970938 14398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39379 } }
I20260811 06:56:22.971167 14398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:22.971319 14398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:22.971498 14398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:22.974170 14398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39379 } }
I20260811 06:56:22.974697 14398 leader_election.cc:302] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46e03c5c9e7a476e99f51bdafa9949f3; no voters:
I20260811 06:56:22.975306 14398 leader_election.cc:288] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:22.975449 14407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:22.976228 14407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [term 1 LEADER]: Becoming Leader. State: Replica: 46e03c5c9e7a476e99f51bdafa9949f3, State: Running, Role: LEADER
I20260811 06:56:22.976768 14407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39379 } }
I20260811 06:56:22.977485 14398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:22.979918 14408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39379 } } }
I20260811 06:56:22.980510 14408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:22.979710 14409 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46e03c5c9e7a476e99f51bdafa9949f3. Latest consensus state: current_term: 1 leader_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46e03c5c9e7a476e99f51bdafa9949f3" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39379 } } }
I20260811 06:56:22.981177 14409 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:22.981487 14418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:22.987676 14418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:22.997473 14418 catalog_manager.cc:1347] Generated new cluster ID: b1b67d3601744a1e9f22e7ffe659992d
I20260811 06:56:22.997581 14418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:23.010342 14418 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:23.012156 14418 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:23.020709 14418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46e03c5c9e7a476e99f51bdafa9949f3: Generated new TSK 0
I20260811 06:56:23.021464 14418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:56:23.501794 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:23.042473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.4.40.62:39379)
0811 06:56:23.042681 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:23.042688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:23.043094 (+ 406us) server_negotiation.cc:407] Connection header received
0811 06:56:23.043318 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:23.043325 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:23.043431 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:23.043610 (+ 179us) server_negotiation.cc:226] Negotiated authn=JWT
0811 06:56:23.044055 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:23.045213 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:23.046599 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:23.047303 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:23.047447 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:23.047918 (+ 471us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 06:56:23.501190 (+453272us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:56:23.501566 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59802: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 06:56:23.501904 14403 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59802: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 06:56:23.502197 14447 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:23.042333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:39379 (local address 127.0.0.1:59802)
0811 06:56:23.042996 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:23.043011 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:23.043125 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:23.043620 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:23.043623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:23.043632 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 06:56:23.043899 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:23.043906 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:23.045365 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:23.045369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:23.046455 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:23.046465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:23.047448 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:23.047491 (+ 43us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:23.047627 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:23.047656 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 06:56:23.501499 (+453843us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 06:56:23.501764 (+ 265us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.40.62:39379: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":386,"threads_started":1}
I20260811 06:56:23.503720 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 14280
I20260811 06:56:23.504921 14390 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:23.609558 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 14280
2026-08-11T06:56:23Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1182 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (424 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (437 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:24 dist-test-slave-7vjv krb5kdc[14473](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:24 dist-test-slave-7vjv krb5kdc[14473](info): set up 2 sockets
Aug 11 06:56:24 dist-test-slave-7vjv krb5kdc[14473](info): commencing operation
krb5kdc: starting...
W20260811 06:56:25.914278 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.394s user 0.000s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:26 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:26.163029 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46803
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:26.494756 14495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:26.495066 14495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:26.495226 14495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:26.502200 14495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:26.502326 14495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:26.502426 14495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:26.502532 14495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:26.502629 14495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:26.512017 14495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:46803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:26.513438 14495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:26.514801 14495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:26.526852 14503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:26.526820 14504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:26.526888 14506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:26.527702 14495 server_base.cc:1034] running on GCE node
I20260811 06:56:26.528551 14495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:26.529907 14495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:26.531113 14495 hybrid_clock.cc:648] HybridClock initialized: now 1786431386531078 us; error 27 us; skew 500 ppm
I20260811 06:56:26.531461 14495 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:26 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431386, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:26.534750 14495 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:26.536216 14495 webserver.cc:460] Webserver started at http://127.4.40.62:37247/ using document root <none> and password file <none>
I20260811 06:56:26.536840 14495 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:26.536942 14495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:26.537220 14495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:26.540203 14495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "dd2bd39806a84f7ea19be827ac850fb1"
format_stamp: "Formatted at 2026-08-11 06:56:26 on dist-test-slave-7vjv"
server_key: "153f620938b9bc54720a3cee44191260"
server_key_iv: "85e5d62790246f8b299f38ef02817983"
server_key_version: "encryptionkey@0"
I20260811 06:56:26.540931 14495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "dd2bd39806a84f7ea19be827ac850fb1"
format_stamp: "Formatted at 2026-08-11 06:56:26 on dist-test-slave-7vjv"
server_key: "153f620938b9bc54720a3cee44191260"
server_key_iv: "85e5d62790246f8b299f38ef02817983"
server_key_version: "encryptionkey@0"
I20260811 06:56:26.545974 14495 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:26.549131 14519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:26.550349 14495 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 06:56:26.550513 14495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "dd2bd39806a84f7ea19be827ac850fb1"
format_stamp: "Formatted at 2026-08-11 06:56:26 on dist-test-slave-7vjv"
server_key: "153f620938b9bc54720a3cee44191260"
server_key_iv: "85e5d62790246f8b299f38ef02817983"
server_key_version: "encryptionkey@0"
I20260811 06:56:26.550716 14495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:26.570593 14495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:26.574458 14495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:26.574762 14495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:26.593710 14495 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:46803
I20260811 06:56:26.593753 14612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:46803 every 8 connection(s)
I20260811 06:56:26.595105 14495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:26.598905 14613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:26.605115 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 14495
I20260811 06:56:26.605335 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:26.606248 4256 external_mini_cluster.cc:1428] Setting key 3f1548231293967e582016c46e33384a
I20260811 06:56:26.610419 14613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1: Bootstrap starting.
I20260811 06:56:26.614236 14613 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:26.615338 14613 log.cc:824] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:26.618417 14613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1: No bootstrap required, opened a new log
Aug 11 06:56:26 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431386, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:26.626621 14613 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2bd39806a84f7ea19be827ac850fb1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46803 } }
I20260811 06:56:26.627012 14613 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:26.627146 14613 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd2bd39806a84f7ea19be827ac850fb1, State: Initialized, Role: FOLLOWER
I20260811 06:56:26.627720 14613 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2bd39806a84f7ea19be827ac850fb1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46803 } }
I20260811 06:56:26.628002 14613 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:26.628189 14613 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:26.628379 14613 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:26.630265 14616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:26.608451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50030 (local address 127.4.40.62:46803)
0811 06:56:26.609258 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:26.609269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:26.609311 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:56:26.611080 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:26.611101 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:26.611438 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:26.611747 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:26.613755 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:26.615149 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:26.616796 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:26.617433 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:26.617574 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:26.622311 (+ 4737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:26.622318 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:26.622334 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:26.622374 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:26.625579 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:26.626265 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:26.626272 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:26.626277 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:26.626436 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:26.626855 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:26.626859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:26.626861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:26.627544 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:26.627904 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:26.628368 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 06:56:26.628854 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":371,"threads_started":1}
I20260811 06:56:26.631348 14613 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2bd39806a84f7ea19be827ac850fb1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46803 } }
I20260811 06:56:26.631840 14613 leader_election.cc:302] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd2bd39806a84f7ea19be827ac850fb1; no voters:
I20260811 06:56:26.632586 14613 leader_election.cc:288] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:26.632721 14618 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:26.633821 14618 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [term 1 LEADER]: Becoming Leader. State: Replica: dd2bd39806a84f7ea19be827ac850fb1, State: Running, Role: LEADER
I20260811 06:56:26.634384 14618 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2bd39806a84f7ea19be827ac850fb1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46803 } }
I20260811 06:56:26.635188 14613 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:26.637251 14619 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd2bd39806a84f7ea19be827ac850fb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2bd39806a84f7ea19be827ac850fb1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46803 } } }
I20260811 06:56:26.637732 14619 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:26.638209 14620 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd2bd39806a84f7ea19be827ac850fb1. Latest consensus state: current_term: 1 leader_uuid: "dd2bd39806a84f7ea19be827ac850fb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd2bd39806a84f7ea19be827ac850fb1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46803 } } }
I20260811 06:56:26.638620 14620 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:26.645418 14630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:26.649777 14630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:26.658474 14630 catalog_manager.cc:1347] Generated new cluster ID: bfa210059b07422fa87280fdcbbb8c86
I20260811 06:56:26.658587 14630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:26.674896 14630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:26.676278 14630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:26.683715 14630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd2bd39806a84f7ea19be827ac850fb1: Generated new TSK 0
I20260811 06:56:26.684547 14630 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:56:26.748075 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46803
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:27.085284 14650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:27.085562 14650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:27.085661 14650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:27.092514 14650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:27.092608 14650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:27.092731 14650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:27.102878 14650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:27.103988 14650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:27.105270 14650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:27.118095 14659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:27.118211 14664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:27.118240 14661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:27.119073 14650 server_base.cc:1034] running on GCE node
I20260811 06:56:27.119702 14650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:27.120671 14650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:27.121891 14650 hybrid_clock.cc:648] HybridClock initialized: now 1786431387121845 us; error 60 us; skew 500 ppm
I20260811 06:56:27.122190 14650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:27 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431387, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:27.125524 14650 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:56:27.126868 14650 webserver.cc:460] Webserver started at http://127.4.40.1:41125/ using document root <none> and password file <none>
I20260811 06:56:27.127516 14650 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:27.127620 14650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:27.127897 14650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:27.130995 14650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "780ff465386b46f8866532cc8bfa1052"
format_stamp: "Formatted at 2026-08-11 06:56:27 on dist-test-slave-7vjv"
server_key: "cb0bbf70cd07f57af4670ce202a02fdd"
server_key_iv: "ee28a33015a6b2748feb49b20ee87099"
server_key_version: "encryptionkey@0"
I20260811 06:56:27.131739 14650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "780ff465386b46f8866532cc8bfa1052"
format_stamp: "Formatted at 2026-08-11 06:56:27 on dist-test-slave-7vjv"
server_key: "cb0bbf70cd07f57af4670ce202a02fdd"
server_key_iv: "ee28a33015a6b2748feb49b20ee87099"
server_key_version: "encryptionkey@0"
I20260811 06:56:27.137060 14650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:56:27.140537 14681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:27.141855 14650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:56:27.142035 14650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "780ff465386b46f8866532cc8bfa1052"
format_stamp: "Formatted at 2026-08-11 06:56:27 on dist-test-slave-7vjv"
server_key: "cb0bbf70cd07f57af4670ce202a02fdd"
server_key_iv: "ee28a33015a6b2748feb49b20ee87099"
server_key_version: "encryptionkey@0"
I20260811 06:56:27.142242 14650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:27.158931 14650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:27.162755 14650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:27.163065 14650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:27.163947 14650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:27.165282 14650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:27.165354 14650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:27.165438 14650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:27.165483 14650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:27.199194 14650 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:45499
I20260811 06:56:27.199277 14881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:45499 every 8 connection(s)
I20260811 06:56:27.200548 14650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:27.203846 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 14650
I20260811 06:56:27.204078 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:27.204826 4256 external_mini_cluster.cc:1428] Setting key e121955ae72ddf50de4d26c8288a05f7
Aug 11 06:56:27 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431387, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:27.222095 14892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:27.203023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:44783 (local address 127.4.40.62:46803)
0811 06:56:27.203574 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:27.203581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:27.204631 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 06:56:27.206922 (+ 2291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:27.206929 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:27.207014 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:27.207126 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:27.208719 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.209873 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.211385 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.212118 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.212286 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:27.215681 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:27.215690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:27.215698 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:27.215742 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:27.218827 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:27.219795 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:27.219799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:27.219802 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:27.219919 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:27.220415 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:27.220418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:27.220420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:27.220680 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:27.220798 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:27.221669 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 06:56:27.221838 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":234,"threads_started":1}
I20260811 06:56:27.223717 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:27.203556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46803 (local address 127.4.40.1:44783)
0811 06:56:27.204429 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:27.204494 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:27.206582 (+ 2088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:27.207358 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:27.207365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:27.207870 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:27.208509 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:27.208526 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.210011 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.210014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:27.211237 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:27.211248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.212344 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.212348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:27.212567 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:27.213283 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:27.213304 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:27.215456 (+ 2152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:27.218969 (+ 3513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:27.218980 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:27.218999 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:27.219678 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:27.220029 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:27.220036 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:27.220043 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:27.220308 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:27.220812 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:27.220823 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:27.221254 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 06:56:27.221569 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 06:56:27.222196 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":372,"threads_started":1}
I20260811 06:56:27.225395 14883 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46803
I20260811 06:56:27.225723 14883 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:27.226476 14883 heartbeater.cc:511] Master 127.4.40.62:46803 requested a full tablet report, sending...
I20260811 06:56:27.228628 14552 ts_manager.cc:194] Registered new tserver with Master: 780ff465386b46f8866532cc8bfa1052 (127.4.40.1:45499)
I20260811 06:56:27.230162 14552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:44783
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:56:27.265156 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46803
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:27.597502 14898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:27.597836 14898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:27.598019 14898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:27.605521 14898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:27.605686 14898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:27.605906 14898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:27.615803 14898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:27.617167 14898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:27.618592 14898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:27.632333 14910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:27.632406 14905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:27.632400 14907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:27.633241 14898 server_base.cc:1034] running on GCE node
I20260811 06:56:27.633893 14898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:27.634865 14898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:27.636129 14898 hybrid_clock.cc:648] HybridClock initialized: now 1786431387636035 us; error 44 us; skew 500 ppm
I20260811 06:56:27.636478 14898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:27 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431387, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:27.639609 14898 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:56:27.641060 14898 webserver.cc:460] Webserver started at http://127.4.40.2:40205/ using document root <none> and password file <none>
I20260811 06:56:27.641741 14898 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:27.641888 14898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:27.642215 14898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:27.645309 14898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "ba5b0c3ce72f4ccbb78f4c5e7a09e7a1"
format_stamp: "Formatted at 2026-08-11 06:56:27 on dist-test-slave-7vjv"
server_key: "3f07a035696e1f2a201fb409ad43d907"
server_key_iv: "05e880b9def6b57a622912f60d0cf349"
server_key_version: "encryptionkey@0"
I20260811 06:56:27.646178 14898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "ba5b0c3ce72f4ccbb78f4c5e7a09e7a1"
format_stamp: "Formatted at 2026-08-11 06:56:27 on dist-test-slave-7vjv"
server_key: "3f07a035696e1f2a201fb409ad43d907"
server_key_iv: "05e880b9def6b57a622912f60d0cf349"
server_key_version: "encryptionkey@0"
I20260811 06:56:27.651245 14898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:56:27.654851 14921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:27.656246 14898 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:56:27.656414 14898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "ba5b0c3ce72f4ccbb78f4c5e7a09e7a1"
format_stamp: "Formatted at 2026-08-11 06:56:27 on dist-test-slave-7vjv"
server_key: "3f07a035696e1f2a201fb409ad43d907"
server_key_iv: "05e880b9def6b57a622912f60d0cf349"
server_key_version: "encryptionkey@0"
I20260811 06:56:27.656620 14898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:27.677779 14898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:27.681747 14898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:27.682057 14898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:27.683030 14898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:27.684496 14898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:27.684608 14898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:27.684762 14898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:27.684835 14898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:27.720983 14898 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:42751
I20260811 06:56:27.721042 15115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:42751 every 8 connection(s)
I20260811 06:56:27.722249 14898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:27.727983 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 14898
I20260811 06:56:27.728179 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:27.728621 4256 external_mini_cluster.cc:1428] Setting key 152d8a1f434435000a359e238769f32d
Aug 11 06:56:27 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431387, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:27.743973 15121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:27.724867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:49211 (local address 127.4.40.62:46803)
0811 06:56:27.725342 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:27.725346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:27.726277 (+ 931us) server_negotiation.cc:407] Connection header received
0811 06:56:27.728161 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:27.728169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:27.728283 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:27.728420 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:27.730013 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.730799 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.732346 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.732794 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.732931 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:27.737019 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:27.737024 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:27.737028 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:27.737060 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:27.740444 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:27.741462 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:27.741472 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:27.741475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:27.741611 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:27.742062 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:27.742067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:27.742070 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:27.742352 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:27.742515 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:27.743303 (+ 788us) server_negotiation.cc:299] Negotiation successful
0811 06:56:27.743541 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":273,"threads_started":1}
I20260811 06:56:27.745090 15122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:27.725387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46803 (local address 127.4.40.2:49211)
0811 06:56:27.726091 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:27.726156 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:27.727877 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:27.728648 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:27.728655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:27.729180 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:27.729832 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:27.729854 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.730963 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.730967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:27.732199 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:27.732212 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:27.733018 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:27.733025 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:27.733274 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:27.733990 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:27.734012 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:27.736793 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:27.740593 (+ 3800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:27.740604 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:27.740623 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:27.741306 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:27.741737 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:27.741741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:27.741747 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:27.741936 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:27.742496 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:27.742504 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:27.742896 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 06:56:27.743214 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 06:56:27.743801 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":276,"threads_started":1}
I20260811 06:56:27.746631 15116 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46803
I20260811 06:56:27.746973 15116 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:27.748016 15116 heartbeater.cc:511] Master 127.4.40.62:46803 requested a full tablet report, sending...
I20260811 06:56:27.749758 14552 ts_manager.cc:194] Registered new tserver with Master: ba5b0c3ce72f4ccbb78f4c5e7a09e7a1 (127.4.40.2:42751)
I20260811 06:56:27.751086 14552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:49211
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:56:27.789364 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46803
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:28.124838 15125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:28.125103 15125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:28.125200 15125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:28.132555 15125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:28.132658 15125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:28.132767 15125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:28.142925 15125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:28.144390 15125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:28.145768 15125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:28.161520 15139 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:28.161553 15142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:28.161782 15137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:28.162331 15125 server_base.cc:1034] running on GCE node
I20260811 06:56:28.163017 15125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:28.163923 15125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:28.165162 15125 hybrid_clock.cc:648] HybridClock initialized: now 1786431388165120 us; error 53 us; skew 500 ppm
I20260811 06:56:28.165477 15125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.168664 15125 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:56:28.170109 15125 webserver.cc:460] Webserver started at http://127.4.40.3:38043/ using document root <none> and password file <none>
I20260811 06:56:28.170790 15125 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:28.170944 15125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:28.171268 15125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:28.174261 15125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "eda27081f4c2441186742fcb4ce54526"
format_stamp: "Formatted at 2026-08-11 06:56:28 on dist-test-slave-7vjv"
server_key: "88a04a6427fb9df712eb5d31349d22eb"
server_key_iv: "2c91f6a2ace44c93ad617be7cd26b124"
server_key_version: "encryptionkey@0"
I20260811 06:56:28.175050 15125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "eda27081f4c2441186742fcb4ce54526"
format_stamp: "Formatted at 2026-08-11 06:56:28 on dist-test-slave-7vjv"
server_key: "88a04a6427fb9df712eb5d31349d22eb"
server_key_iv: "2c91f6a2ace44c93ad617be7cd26b124"
server_key_version: "encryptionkey@0"
I20260811 06:56:28.179970 15125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 06:56:28.183375 15152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:28.184679 15125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:28.184806 15125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "eda27081f4c2441186742fcb4ce54526"
format_stamp: "Formatted at 2026-08-11 06:56:28 on dist-test-slave-7vjv"
server_key: "88a04a6427fb9df712eb5d31349d22eb"
server_key_iv: "2c91f6a2ace44c93ad617be7cd26b124"
server_key_version: "encryptionkey@0"
I20260811 06:56:28.184973 15125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:28.205215 15125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:28.209271 15125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:28.209525 15125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:28.210424 15125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:28.211730 15125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:28.211808 15125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:28.211891 15125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:28.211936 15125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:28.233180 14883 heartbeater.cc:503] Master 127.4.40.62:46803 was elected leader, sending a full tablet report...
I20260811 06:56:28.247910 15125 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:33387
I20260811 06:56:28.247996 15357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:33387 every 8 connection(s)
I20260811 06:56:28.249204 15125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:28.254097 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 15125
I20260811 06:56:28.254271 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:28.254858 4256 external_mini_cluster.cc:1428] Setting key a28a604e0dd1b7dd38c1771b1eb708c1
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:28.269266 15362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.251640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:34069 (local address 127.4.40.62:46803)
0811 06:56:28.252070 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.252074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.253145 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 06:56:28.255627 (+ 2482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.255632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.255738 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.255905 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.257582 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.258490 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.260039 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.260497 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.260644 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.263861 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.263866 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.263869 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.263900 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.266349 (+ 2449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.267000 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.267004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.267006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.267126 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.267512 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.267515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.267517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.267753 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.267863 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.268658 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.268865 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":219,"threads_started":1}
I20260811 06:56:28.270255 15363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.252253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46803 (local address 127.4.40.3:34069)
0811 06:56:28.252978 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:28.253021 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:28.255360 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:28.256199 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:28.256210 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:28.256788 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:28.257406 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:28.257423 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.258645 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.258649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:28.259867 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:28.259879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.260735 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.260743 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:28.260979 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.261587 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:28.261611 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:28.263684 (+ 2073us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:28.266474 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:28.266480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:28.266491 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:28.266901 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:28.267221 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:28.267224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:28.267227 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:28.267402 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:28.267854 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:28.267859 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:28.268253 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 06:56:28.268510 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 06:56:28.269008 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":288,"threads_started":1}
I20260811 06:56:28.271564 15359 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46803
I20260811 06:56:28.271863 15359 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:28.272640 15359 heartbeater.cc:511] Master 127.4.40.62:46803 requested a full tablet report, sending...
I20260811 06:56:28.274119 14552 ts_manager.cc:194] Registered new tserver with Master: eda27081f4c2441186742fcb4ce54526 (127.4.40.3:33387)
I20260811 06:56:28.274946 14552 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:34069
I20260811 06:56:28.282281 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/test-admin.keytab.
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.318939 4256 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 06:56:28.324537 15371 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:28.322022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.4.40.1:45499)
0811 06:56:28.322650 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.322658 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.322685 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:56:28.322749 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.322756 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.322994 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.323128 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.324018 (+ 890us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40816: BlockingRecv error: recv got EOF from 127.0.0.1:40816 (error 108)
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":315,"threads_started":1}
W20260811 06:56:28.324580 4256 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.326699 4256 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.328346 4256 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:28.347692 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.330039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40828 (local address 127.4.40.1:45499)
0811 06:56:28.330274 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.330281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.330579 (+ 298us) server_negotiation.cc:407] Connection header received
0811 06:56:28.330840 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.330844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.330910 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.331038 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.332147 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.333156 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.334452 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.335231 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.335391 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.340285 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.340308 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.340322 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.340374 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.344764 (+ 4390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.345544 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.345552 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.345554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.345675 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.346065 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.346070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.346075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.346509 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.346728 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.347122 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.347386 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 06:56:28.353607 15371 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:28.351806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.4.40.1:45499)
0811 06:56:28.352015 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.352020 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.352271 (+ 251us) server_negotiation.cc:407] Connection header received
0811 06:56:28.352455 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.352460 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.352542 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.352645 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.353365 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40832: BlockingRecv error: recv got EOF from 127.0.0.1:40832 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 06:56:28.353853 4256 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.355614 4256 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.356942 4256 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:28.370916 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.358186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40840 (local address 127.4.40.1:45499)
0811 06:56:28.358398 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.358403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.358654 (+ 251us) server_negotiation.cc:407] Connection header received
0811 06:56:28.358771 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.358775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.358826 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.358900 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.359836 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.360699 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.362010 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.362421 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.362524 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.366088 (+ 3564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.366093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.366096 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.366125 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.368762 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.369346 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.369349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.369351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.369464 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.369761 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.369764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.369766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.370011 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.370193 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.370442 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.370643 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 06:56:28.375588 15371 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:28.373862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40850 (local address 127.4.40.1:45499)
0811 06:56:28.374086 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.374094 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.374247 (+ 153us) server_negotiation.cc:407] Connection header received
0811 06:56:28.374359 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.374362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.374430 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.374506 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.375352 (+ 846us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40850: BlockingRecv error: recv got EOF from 127.0.0.1:40850 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 06:56:28.375663 4256 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786431333861527-4256-0/krb5kdc/krb5cc' of type FILE
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.377275 4256 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:28.378537 4256 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:28.392069 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.379783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40860 (local address 127.4.40.1:45499)
0811 06:56:28.379933 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.379937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.380368 (+ 431us) server_negotiation.cc:407] Connection header received
0811 06:56:28.380454 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.380457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.380513 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.380596 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.381489 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.382204 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.383358 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.383777 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.383885 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.387197 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.387203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.387206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.387235 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.389948 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.390533 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.390537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.390539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.390653 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.390941 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.390944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.390946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.391211 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.391436 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.391647 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.391799 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 06:56:28.393704 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 14650
I20260811 06:56:28.405287 15371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.394372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40876 (local address 127.4.40.1:45499)
0811 06:56:28.394520 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.394524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.394907 (+ 383us) server_negotiation.cc:407] Connection header received
0811 06:56:28.395034 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.395037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.395087 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.395161 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.396145 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.396814 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.397954 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.398392 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.398485 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.400497 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.400502 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.400505 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.400534 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.403065 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.403638 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.403641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.403644 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.403772 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.404084 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.404087 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.404089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.404393 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.404598 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.404847 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.405025 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:56:28.405885 14875 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:28.492022 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 14650
I20260811 06:56:28.520849 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 14898
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:56:28.535598 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.521754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53318 (local address 127.4.40.2:42751)
0811 06:56:28.522288 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.522295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.522321 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:56:28.522379 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.522386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.522591 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.522712 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.523674 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.524837 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.526116 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.526650 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.526745 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.530503 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.530514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.530527 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.530566 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.533205 (+ 2639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.533799 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.533805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.533808 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.533924 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.534230 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.534232 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.534234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.534543 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.534767 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.535137 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.535324 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":262,"threads_started":1}
I20260811 06:56:28.536979 15111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:28.619571 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 14898
I20260811 06:56:28.650236 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 15125
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[14473](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431388, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:56:28.665793 15405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:28.651220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39472 (local address 127.4.40.3:33387)
0811 06:56:28.651854 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:28.651861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:28.651893 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:56:28.651956 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:28.651962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:28.652247 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:28.652393 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:28.653459 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.654493 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.655848 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:28.656505 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:28.656606 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:28.660501 (+ 3895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:28.660513 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:28.660526 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:28.660555 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:28.663133 (+ 2578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.663754 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.663761 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.663763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.663888 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:28.664338 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:28.664343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:28.664345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:28.664700 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:28.664992 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:28.665179 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 06:56:28.665477 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":282,"threads_started":1}
I20260811 06:56:28.667110 15347 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:28.751221 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 15125
I20260811 06:56:28.779894 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 14495
I20260811 06:56:28.781134 14604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:28.885349 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 14495
2026-08-11T06:56:28Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4415 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:28 dist-test-slave-7vjv krb5kdc[15418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:29 dist-test-slave-7vjv krb5kdc[15418](info): set up 2 sockets
Aug 11 06:56:29 dist-test-slave-7vjv krb5kdc[15418](info): commencing operation
krb5kdc: starting...
W20260811 06:56:30.254225 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.003s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:30 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:30Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab.
Entry for principal oryx/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab.
I20260811 06:56:30.511495 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45025
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab
--principal=oryx/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45025
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:30.850503 15434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:30.850798 15434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:30.850900 15434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:30.859205 15434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:30.859333 15434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:30.859411 15434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:30.859486 15434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:30.859536 15434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:30.869690 15434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab
--principal=oryx/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:30.871127 15434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:30.872552 15434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:30.885865 15449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:30.886075 15448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:30.886092 15451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:30.889565 15434 server_base.cc:1034] running on GCE node
I20260811 06:56:30.890352 15434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:30.891747 15434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:30.892987 15434 hybrid_clock.cc:648] HybridClock initialized: now 1786431390892945 us; error 43 us; skew 500 ppm
I20260811 06:56:30.893338 15434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:30 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:30.896588 15434 init.cc:377] Logged in from keytab as oryx/127.4.40.62@KRBTEST.COM (short username oryx)
I20260811 06:56:30.898059 15434 webserver.cc:460] Webserver started at http://127.4.40.62:35551/ using document root <none> and password file <none>
I20260811 06:56:30.898715 15434 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:30.898818 15434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:30.899078 15434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:30.902051 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "d0c6a3f5057249c1b2cf914883ac53c9"
format_stamp: "Formatted at 2026-08-11 06:56:30 on dist-test-slave-7vjv"
server_key: "aa8fbbf28e0b8f880e8197dc8cccf94c"
server_key_iv: "0396c73ff67599156e3b42ccb6ad0953"
server_key_version: "encryptionkey@0"
I20260811 06:56:30.902745 15434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "d0c6a3f5057249c1b2cf914883ac53c9"
format_stamp: "Formatted at 2026-08-11 06:56:30 on dist-test-slave-7vjv"
server_key: "aa8fbbf28e0b8f880e8197dc8cccf94c"
server_key_iv: "0396c73ff67599156e3b42ccb6ad0953"
server_key_version: "encryptionkey@0"
I20260811 06:56:30.907658 15434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:56:30.910902 15461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:30.912220 15434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:30.912400 15434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "d0c6a3f5057249c1b2cf914883ac53c9"
format_stamp: "Formatted at 2026-08-11 06:56:30 on dist-test-slave-7vjv"
server_key: "aa8fbbf28e0b8f880e8197dc8cccf94c"
server_key_iv: "0396c73ff67599156e3b42ccb6ad0953"
server_key_version: "encryptionkey@0"
I20260811 06:56:30.912585 15434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:30.928488 15434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:30.933352 15434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:30.933665 15434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:30.952702 15434 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45025
I20260811 06:56:30.952744 15548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45025 every 8 connection(s)
I20260811 06:56:30.954080 15434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:30.955364 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 15434
I20260811 06:56:30.955590 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:30.956508 4256 external_mini_cluster.cc:1428] Setting key 80a591d8a421a5a224abbdf6a6e6d366
I20260811 06:56:30.958863 15549 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:56:30 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:30.972218 15549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Bootstrap starting.
I20260811 06:56:30.975544 15549 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:30.976630 15549 log.cc:824] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:30.979317 15555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:30.958587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41026 (local address 127.4.40.62:45025)
0811 06:56:30.959400 (+ 813us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:30.959412 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:30.959452 (+ 40us) server_negotiation.cc:407] Connection header received
0811 06:56:30.961096 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:30.961118 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:30.961486 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:30.961803 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:30.963604 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:30.965009 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:30.966772 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:30.967391 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:30.967532 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:30.972247 (+ 4715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:30.972256 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:30.972269 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:30.972308 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:30.975307 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:30.976042 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:30.976055 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:30.976061 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:30.976284 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:30.976671 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:30.976677 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:30.976680 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:30.977373 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:30.977723 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:30.978013 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 06:56:30.978391 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":407,"threads_started":1}
I20260811 06:56:30.979702 15549 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: No bootstrap required, opened a new log
I20260811 06:56:30.987920 15549 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } }
I20260811 06:56:30.988322 15549 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:30.988401 15549 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0c6a3f5057249c1b2cf914883ac53c9, State: Initialized, Role: FOLLOWER
I20260811 06:56:30.988991 15549 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } }
I20260811 06:56:30.989197 15549 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:30.989290 15549 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:30.989418 15549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:30.991798 15549 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } }
I20260811 06:56:30.992321 15549 leader_election.cc:302] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0c6a3f5057249c1b2cf914883ac53c9; no voters:
I20260811 06:56:30.992883 15549 leader_election.cc:288] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:30.993043 15559 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:30.993710 15559 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 LEADER]: Becoming Leader. State: Replica: d0c6a3f5057249c1b2cf914883ac53c9, State: Running, Role: LEADER
I20260811 06:56:30.994207 15559 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } }
I20260811 06:56:30.994915 15549 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:30.998004 15561 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c6a3f5057249c1b2cf914883ac53c9. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } }
I20260811 06:56:30.997918 15560 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } }
I20260811 06:56:30.998355 15560 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:30.998355 15561 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:31.001379 15575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:31.006203 15575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:31.014582 15575 catalog_manager.cc:1347] Generated new cluster ID: 1d9658236d6049c2b98b8031f9e4b061
I20260811 06:56:31.014696 15575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:31.024677 15575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:31.026542 15575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:31.036607 15575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Generated new TSK 0
I20260811 06:56:31.037549 15575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab.
Entry for principal oryx/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab.
I20260811 06:56:31.103330 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45025
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:31.453017 15595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:31.453305 15595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:31.453480 15595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:31.460594 15595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:31.460741 15595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:31.460935 15595 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:31.470291 15595 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:31.471562 15595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:31.472947 15595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:31.486053 15604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:31.486483 15605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:31.486622 15608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:31.487507 15595 server_base.cc:1034] running on GCE node
I20260811 06:56:31.488253 15595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:31.489180 15595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:31.490451 15595 hybrid_clock.cc:648] HybridClock initialized: now 1786431391490391 us; error 48 us; skew 500 ppm
I20260811 06:56:31.490769 15595 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:31 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431391, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:31.494050 15595 init.cc:377] Logged in from keytab as oryx/127.4.40.1@KRBTEST.COM (short username oryx)
I20260811 06:56:31.495527 15595 webserver.cc:460] Webserver started at http://127.4.40.1:37679/ using document root <none> and password file <none>
I20260811 06:56:31.496300 15595 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:31.496456 15595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:31.496796 15595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:31.499888 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "04bed8119bb44f1580b28e08aa5132ac"
format_stamp: "Formatted at 2026-08-11 06:56:31 on dist-test-slave-7vjv"
server_key: "56575d6f6cfbeabd917ab9a0a1c1882b"
server_key_iv: "47c0ba7bc423613821f18740d863d49c"
server_key_version: "encryptionkey@0"
I20260811 06:56:31.500695 15595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "04bed8119bb44f1580b28e08aa5132ac"
format_stamp: "Formatted at 2026-08-11 06:56:31 on dist-test-slave-7vjv"
server_key: "56575d6f6cfbeabd917ab9a0a1c1882b"
server_key_iv: "47c0ba7bc423613821f18740d863d49c"
server_key_version: "encryptionkey@0"
I20260811 06:56:31.505741 15595 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:31.509157 15617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:31.510586 15595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 06:56:31.510725 15595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "04bed8119bb44f1580b28e08aa5132ac"
format_stamp: "Formatted at 2026-08-11 06:56:31 on dist-test-slave-7vjv"
server_key: "56575d6f6cfbeabd917ab9a0a1c1882b"
server_key_iv: "47c0ba7bc423613821f18740d863d49c"
server_key_version: "encryptionkey@0"
I20260811 06:56:31.510895 15595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:31.535295 15595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:31.539335 15595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:31.539594 15595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:31.540545 15595 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:31.541843 15595 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:31.541917 15595 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:31.541996 15595 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:31.542040 15595 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:31.576527 15595 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:46877
I20260811 06:56:31.576604 15799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:46877 every 8 connection(s)
I20260811 06:56:31.577854 15595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:31.579087 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 15595
I20260811 06:56:31.579305 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:31.580219 4256 external_mini_cluster.cc:1428] Setting key 7c7d774546d1c097bb50938a8beba201
Aug 11 06:56:31 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431391, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:31.598807 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:31.580761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:46737 (local address 127.4.40.62:45025)
0811 06:56:31.581246 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:31.581250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:31.583855 (+ 2605us) server_negotiation.cc:407] Connection header received
0811 06:56:31.584488 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:31.584493 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:31.584581 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:31.584710 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:31.586184 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:31.586887 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:31.588515 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:31.588914 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:31.589052 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:31.592855 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:31.592860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:31.592863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:31.592893 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:31.595676 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:31.596514 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:31.596518 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:31.596520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:31.596639 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:31.597088 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:31.597094 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:31.597097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:31.597363 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:31.597520 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:31.598206 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 06:56:31.598432 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":228,"threads_started":1}
I20260811 06:56:31.599804 15805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:31.581331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:46737)
0811 06:56:31.582224 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:31.582291 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:31.584195 (+ 1904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:31.584940 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:31.584948 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:31.585461 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:31.586013 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:31.586031 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:31.587060 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:31.587066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:31.588362 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:31.588374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:31.589157 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:31.589165 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:31.589387 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:31.590063 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:31.590085 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:31.592645 (+ 2560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:31.595813 (+ 3168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:31.595822 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:31.595837 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:31.596337 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:31.596757 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:31.596764 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:31.596767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:31.596961 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:31.597500 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:31.597508 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:31.597822 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 06:56:31.598084 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 06:56:31.598615 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":335,"threads_started":1}
I20260811 06:56:31.601543 15800 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45025
I20260811 06:56:31.601910 15800 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:31.602797 15800 heartbeater.cc:511] Master 127.4.40.62:45025 requested a full tablet report, sending...
I20260811 06:56:31.605201 15499 ts_manager.cc:194] Registered new tserver with Master: 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877)
I20260811 06:56:31.607470 15499 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:46737
WARNING: no policy specified for oryx/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab.
Entry for principal oryx/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab.
I20260811 06:56:31.642822 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45025
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:31.982718 15811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:31.982957 15811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:31.983047 15811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:31.990247 15811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:31.990345 15811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:31.990476 15811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:31.999871 15811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:32.001152 15811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:32.002497 15811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:32.015976 15817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:32.015928 15821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:32.016031 15819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:32.017174 15811 server_base.cc:1034] running on GCE node
I20260811 06:56:32.017732 15811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:32.018616 15811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:32.019819 15811 hybrid_clock.cc:648] HybridClock initialized: now 1786431392019779 us; error 26 us; skew 500 ppm
I20260811 06:56:32.020141 15811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:32 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431392, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:32.023159 15811 init.cc:377] Logged in from keytab as oryx/127.4.40.2@KRBTEST.COM (short username oryx)
I20260811 06:56:32.024633 15811 webserver.cc:460] Webserver started at http://127.4.40.2:36201/ using document root <none> and password file <none>
I20260811 06:56:32.025296 15811 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:32.025427 15811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:32.025743 15811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:32.028671 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "e0cb6e18a37446a4ba804fb73c3077ac"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "27914e2b9e9dfffb75b56a3284b0219d"
server_key_iv: "b6a104d486420b5969531891e12dae24"
server_key_version: "encryptionkey@0"
I20260811 06:56:32.029469 15811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "e0cb6e18a37446a4ba804fb73c3077ac"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "27914e2b9e9dfffb75b56a3284b0219d"
server_key_iv: "b6a104d486420b5969531891e12dae24"
server_key_version: "encryptionkey@0"
I20260811 06:56:32.034551 15811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:32.038058 15836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:32.039307 15811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:56:32.039474 15811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "e0cb6e18a37446a4ba804fb73c3077ac"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "27914e2b9e9dfffb75b56a3284b0219d"
server_key_iv: "b6a104d486420b5969531891e12dae24"
server_key_version: "encryptionkey@0"
I20260811 06:56:32.039680 15811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:32.052315 15811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:32.056172 15811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:32.056456 15811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:32.057356 15811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:32.058640 15811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:32.058714 15811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:32.058884 15811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:32.058940 15811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:32.092722 15811 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:44375
I20260811 06:56:32.092767 16040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:44375 every 8 connection(s)
I20260811 06:56:32.093959 15811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:32.098701 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 15811
I20260811 06:56:32.098884 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:32.099481 4256 external_mini_cluster.cc:1428] Setting key 0dbb6401b4b7d5d15f9f4018ae9a0bb7
Aug 11 06:56:32 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431392, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:32.114530 15804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:32.096503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:39569 (local address 127.4.40.62:45025)
0811 06:56:32.096694 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:32.096698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:32.097862 (+ 1164us) server_negotiation.cc:407] Connection header received
0811 06:56:32.100280 (+ 2418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:32.100288 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:32.100387 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:32.100535 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:32.102313 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.103030 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.104607 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.105049 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.105189 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:32.108808 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:32.108814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:32.108817 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:32.108848 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:32.111410 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:32.112179 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:32.112192 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:32.112195 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:32.112326 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:32.112771 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:32.112774 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:32.112777 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:32.113020 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:32.113176 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:32.113876 (+ 700us) server_negotiation.cc:299] Negotiation successful
0811 06:56:32.114168 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:56:32.115643 16043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:32.096996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:39569)
0811 06:56:32.097710 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:32.097746 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:32.099944 (+ 2198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:32.100756 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:32.100784 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:32.101383 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:32.102117 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:32.102137 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.103186 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.103190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:32.104457 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:32.104470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.105282 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.105289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:32.105519 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:32.106199 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:32.106221 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:32.108589 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:32.111597 (+ 3008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:32.111605 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:32.111620 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:32.112029 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:32.112444 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:32.112449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:32.112453 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:32.112642 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:32.113151 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:32.113158 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:32.113526 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 06:56:32.113776 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 06:56:32.114347 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":290,"threads_started":1}
I20260811 06:56:32.117218 16041 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45025
I20260811 06:56:32.117514 16041 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:32.118219 16041 heartbeater.cc:511] Master 127.4.40.62:45025 requested a full tablet report, sending...
I20260811 06:56:32.119799 15499 ts_manager.cc:194] Registered new tserver with Master: e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375)
I20260811 06:56:32.121053 15499 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:39569
WARNING: no policy specified for oryx/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab.
Entry for principal oryx/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab.
I20260811 06:56:32.158664 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45025
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:32.514235 16047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:32.514518 16047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:32.514698 16047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:32.522446 16047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:32.522553 16047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:32.522779 16047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:32.532004 16047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45025
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:32.533300 16047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:32.534601 16047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:32.547864 16056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:32.547864 16060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:32.547899 16063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:32.548430 16047 server_base.cc:1034] running on GCE node
I20260811 06:56:32.549321 16047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:32.550246 16047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:32.551493 16047 hybrid_clock.cc:648] HybridClock initialized: now 1786431392551429 us; error 50 us; skew 500 ppm
I20260811 06:56:32.551755 16047 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:32 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431392, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:32.554816 16047 init.cc:377] Logged in from keytab as oryx/127.4.40.3@KRBTEST.COM (short username oryx)
I20260811 06:56:32.556272 16047 webserver.cc:460] Webserver started at http://127.4.40.3:42529/ using document root <none> and password file <none>
I20260811 06:56:32.556955 16047 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:32.557111 16047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:32.557405 16047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:32.560351 16047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "78862879db8d48389ead036510fb4094"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "a3171cc979f7fa1d9f928400afbeca57"
server_key_iv: "18d02d49c138a8637ee0154adb24a17f"
server_key_version: "encryptionkey@0"
I20260811 06:56:32.561137 16047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "78862879db8d48389ead036510fb4094"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "a3171cc979f7fa1d9f928400afbeca57"
server_key_iv: "18d02d49c138a8637ee0154adb24a17f"
server_key_version: "encryptionkey@0"
I20260811 06:56:32.566269 16047 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:32.569623 16075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:32.570909 16047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:56:32.571099 16047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "78862879db8d48389ead036510fb4094"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "a3171cc979f7fa1d9f928400afbeca57"
server_key_iv: "18d02d49c138a8637ee0154adb24a17f"
server_key_version: "encryptionkey@0"
I20260811 06:56:32.571305 16047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:32.599512 16047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:32.603937 16047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:32.604277 16047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:32.605280 16047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:32.606668 16047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:32.606783 16047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:32.606940 16047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:32.607028 16047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:32.610484 15800 heartbeater.cc:503] Master 127.4.40.62:45025 was elected leader, sending a full tablet report...
I20260811 06:56:32.641949 16047 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:42639
I20260811 06:56:32.642012 16279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:42639 every 8 connection(s)
I20260811 06:56:32.643267 16047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:32.651597 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 16047
I20260811 06:56:32.651804 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:32.652421 4256 external_mini_cluster.cc:1428] Setting key 893d36e353ddd037b5b8ae2a8594e07d
Aug 11 06:56:32 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431392, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:32.663239 16282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:32.645897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:53025 (local address 127.4.40.62:45025)
0811 06:56:32.646449 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:32.646456 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:32.647343 (+ 887us) server_negotiation.cc:407] Connection header received
0811 06:56:32.649148 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:32.649152 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:32.649220 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:32.649316 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:32.650786 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.651573 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.653745 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.654240 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.654380 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:32.658159 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:32.658164 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:32.658167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:32.658198 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:32.660706 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:32.661363 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:32.661367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:32.661369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:32.661486 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:32.661839 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:32.661843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:32.661845 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:32.662094 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:32.662210 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:32.662805 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 06:56:32.662972 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":297,"threads_started":1}
I20260811 06:56:32.664275 16283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:32.646436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:53025)
0811 06:56:32.647183 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:32.647220 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:32.648891 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:32.649518 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:32.649526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:32.650033 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:32.650620 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:32.650637 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.651741 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.651748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:32.653598 (+ 1850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:32.653611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:32.654515 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:32.654524 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:32.654839 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:32.655579 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:32.655611 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:32.657971 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:32.660827 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:32.660834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:32.660845 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:32.661254 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:32.661573 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:32.661576 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:32.661578 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:32.661741 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:32.662202 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:32.662207 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:32.662493 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 06:56:32.662703 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 06:56:32.663069 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":300,"threads_started":1}
I20260811 06:56:32.665548 16280 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45025
I20260811 06:56:32.665850 16280 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:32.666805 16280 heartbeater.cc:511] Master 127.4.40.62:45025 requested a full tablet report, sending...
I20260811 06:56:32.668586 15499 ts_manager.cc:194] Registered new tserver with Master: 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:32.669384 15499 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:53025
I20260811 06:56:32.680358 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.40.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.61.keytab.
Entry for principal oryx/127.4.40.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.61.keytab.
I20260811 06:56:32.739818 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:40483
--webserver_interface=127.4.40.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.61.keytab
--principal=oryx/127.4.40.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45025,127.4.40.61:40483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:33.084030 16287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:33.084365 16287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:33.084472 16287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:33.091498 16287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:33.091609 16287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:33.091653 16287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:33.091701 16287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:33.091738 16287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:33.101362 16287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45025,127.4.40.61:40483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.61.keytab
--principal=oryx/127.4.40.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:40483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.40.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
W20260811 06:56:33.101720 16287 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.40.62:45025,127.4.40.61:40483'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 06:56:33.102653 16287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:33.103909 16287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:33.115742 16295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:33.115695 16300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:33.115785 16297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:33.116760 16287 server_base.cc:1034] running on GCE node
I20260811 06:56:33.117559 16287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:33.118876 16287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:33.120154 16287 hybrid_clock.cc:648] HybridClock initialized: now 1786431393120039 us; error 58 us; skew 500 ppm
I20260811 06:56:33.120488 16287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:33.123528 16041 heartbeater.cc:503] Master 127.4.40.62:45025 was elected leader, sending a full tablet report...
I20260811 06:56:33.123853 16287 init.cc:377] Logged in from keytab as oryx/127.4.40.61@KRBTEST.COM (short username oryx)
I20260811 06:56:33.125519 16287 webserver.cc:460] Webserver started at http://127.4.40.61:44285/ using document root <none> and password file <none>
I20260811 06:56:33.126212 16287 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:33.126330 16287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:33.126682 16287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:33.129705 16287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/instance:
uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "9e118dd57d6cda60ccbd46762afd6d77"
server_key_iv: "d5037bcc6d5ab0c7df44078dcffa2c0e"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.130484 16287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal/instance:
uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "9e118dd57d6cda60ccbd46762afd6d77"
server_key_iv: "d5037bcc6d5ab0c7df44078dcffa2c0e"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.135628 16287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:56:33.139039 16314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:33.140421 16287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:56:33.140609 16287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "9e118dd57d6cda60ccbd46762afd6d77"
server_key_iv: "d5037bcc6d5ab0c7df44078dcffa2c0e"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.140815 16287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:33.164376 16287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:33.168377 16287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:33.168689 16287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:33.188896 16287 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.61:40483
I20260811 06:56:33.188915 16409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.61:40483 every 8 connection(s)
I20260811 06:56:33.190241 16287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
I20260811 06:56:33.193991 16410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:33.195302 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 16287
I20260811 06:56:33.203373 16410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.61@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:33.220865 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.197627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45212 (local address 127.4.40.61:40483)
0811 06:56:33.198415 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.198426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.198469 (+ 43us) server_negotiation.cc:407] Connection header received
0811 06:56:33.200143 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.200175 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.200547 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.200869 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.202770 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.204255 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.205978 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.207138 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.207361 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.212455 (+ 5094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.212462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.212472 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.212522 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.216314 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.217169 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.217176 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.217182 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.217354 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.217774 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.217781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.217785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.218274 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.218585 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.219034 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.219433 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":380,"threads_started":1}
I20260811 06:56:33.222707 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.206210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.4.40.62:45025)
0811 06:56:33.206721 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.206730 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.207200 (+ 470us) server_negotiation.cc:407] Connection header received
0811 06:56:33.207483 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.207490 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.207589 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.207709 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.208963 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.210200 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.211896 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.212510 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.212656 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.216573 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.216580 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.216586 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.216634 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.219683 (+ 3049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.220543 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.220547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.220549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.220671 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.221185 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.221190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.221193 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.221631 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.221784 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.222232 (+ 448us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.222460 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":262,"threads_started":1}
I20260811 06:56:33.222851 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.206269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41030)
0811 06:56:33.207062 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.207104 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.207304 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.207767 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.207772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.208544 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.208799 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.208808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.210340 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.210344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.211748 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.211761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.212712 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.212719 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.212899 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.213602 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.213623 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.216371 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.219810 (+ 3439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.219820 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.219827 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.220430 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.220787 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.220806 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.220815 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.220995 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.221764 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.221770 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.221911 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.222082 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.222405 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":597,"thread_start_us":443,"threads_started":1}
I20260811 06:56:33.224663 16410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.61@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
I20260811 06:56:33.236340 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.225352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45216)
0811 06:56:33.225552 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.225570 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.225668 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.225971 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.225974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.226272 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.226396 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.226403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.227293 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.227296 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.228233 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.228244 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.228866 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.228870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.228988 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.229542 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.229558 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.231534 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.234255 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.234258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.234260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.234624 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.234921 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.234924 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.234926 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.235057 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.235553 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.235556 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.235685 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.235793 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.235990 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:56:33.236508 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.225613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.4.40.61:40483)
0811 06:56:33.225750 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.225754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.225768 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:56:33.225805 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.225808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.225857 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.225932 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.226515 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.227186 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.228354 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.228738 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.228835 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.231696 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.231700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.231703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.231730 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.234124 (+ 2394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.234720 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.234723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.234725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.234829 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.235173 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.235176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.235178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.235447 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.235554 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.235877 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.236178 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":52}
I20260811 06:56:33.241294 16410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Bootstrap starting.
I20260811 06:56:33.244419 16410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:33.245512 16410 log.cc:824] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:33.248152 16410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: No bootstrap required, opened a new log
I20260811 06:56:33.254833 16410 raft_consensus.cc:348] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } }
I20260811 06:56:33.255131 16410 raft_consensus.cc:374] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:33.255198 16410 raft_consensus.cc:729] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f45dd7db5d6444a998c5afd3ede4fdc2, State: Initialized, Role: FOLLOWER
I20260811 06:56:33.255730 16410 consensus_queue.cc:260] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } }
I20260811 06:56:33.256592 16424 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } } }
I20260811 06:56:33.257000 16424 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:56:33.257627 16410 sys_catalog.cc:564] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:56:33.266353 16439 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:56:33.266484 16439 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:56:33.268939 16287 master_runner.cc:428] Detected that this master f45dd7db5d6444a998c5afd3ede4fdc2 is joining an existing cluster
I20260811 06:56:33.269116 16287 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.40.61:40483
I20260811 06:56:33.291124 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.272904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.4.40.62:45025)
0811 06:56:33.273106 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.273113 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.274227 (+ 1114us) server_negotiation.cc:407] Connection header received
0811 06:56:33.274492 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.274498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.274629 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.274707 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.276531 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.277686 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.279926 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.280650 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.280762 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.283931 (+ 3169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.283938 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.283942 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.283994 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.287656 (+ 3662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.288582 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.288588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.288591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.288761 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.289269 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.289274 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.289278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.289666 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.289809 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.290489 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.290746 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:56:33.292289 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.272966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41032)
0811 06:56:33.274083 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.274134 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.274339 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.275021 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.275027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.275735 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.276348 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.276364 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.277884 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.277891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.279753 (+ 1862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.279771 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.281092 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.281100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.281308 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.282331 (+ 1023us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.282353 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.283714 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.287820 (+ 4106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.287825 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.287829 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.288421 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.288933 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.288939 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.288943 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.289129 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.289911 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.289916 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.290151 (+ 235us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.291401 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.291729 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"thread_start_us":787,"threads_started":1}
I20260811 06:56:33.300858 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.273336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45230 (local address 127.4.40.61:40483)
0811 06:56:33.273554 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.273562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.282171 (+ 8609us) server_negotiation.cc:407] Connection header received
0811 06:56:33.282337 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.282345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.282415 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.282488 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.284611 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.285732 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.287884 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.288756 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.288873 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.291566 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.291575 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.291582 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.291622 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.296222 (+ 4600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.298537 (+ 2315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.298542 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.298546 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.298700 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.299179 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.299184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.299187 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.299562 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.299765 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.300271 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.300505 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:56:33.301939 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.273051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45230)
0811 06:56:33.282050 (+ 8999us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.282070 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.282201 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.282690 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.282697 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.284022 (+ 1325us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.284460 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.284476 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.285951 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.285958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.287714 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.287733 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.289228 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.289235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.289438 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.290112 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.290212 (+ 100us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.291389 (+ 1177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.296743 (+ 5354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.296750 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.297766 (+ 1016us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.298400 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.298832 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.298837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.298840 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.299022 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.299819 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.299823 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.299956 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.301157 (+ 1201us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.301469 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8834,"spinlock_wait_cycles":23936,"thread_start_us":8643,"threads_started":1}
WARNING: no policy specified for oryx/127.4.40.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.60@KRBTEST.COM" created.
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.62@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
I20260811 06:56:33.323567 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.300490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.4.40.61:40483)
0811 06:56:33.301118 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.301123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.305102 (+ 3979us) server_negotiation.cc:407] Connection header received
0811 06:56:33.305501 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.305509 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.305616 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.305726 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.307177 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.308507 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.310488 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.311282 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.311432 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.317637 (+ 6205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.317642 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.317645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.317679 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.320648 (+ 2969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.321395 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.321402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.321405 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.321556 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.321947 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.321953 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.321956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.322235 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.322393 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.322875 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.323124 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":3829,"threads_started":1}
I20260811 06:56:33.323593 16454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.300059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45238)
0811 06:56:33.304980 (+ 4921us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.305021 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.305246 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.305797 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.305804 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.306687 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.307009 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.307022 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.308675 (+ 1653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.308681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.310322 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.310339 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.311472 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.311479 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.311700 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.312564 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.312591 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.317415 (+ 4824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.320796 (+ 3381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.320810 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.320818 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.321266 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.321676 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.321681 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.321685 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.321832 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.322374 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.322383 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.322592 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.322783 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.323125 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4656,"thread_start_us":4485,"threads_started":1}
I20260811 06:56:33.325834 15499 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.40.61:40483: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
member_type: NON_VOTER
last_known_addr {
host: "127.4.40.61"
port: 40483
}
attrs {
promote: true
}
}
dest_uuid: "d0c6a3f5057249c1b2cf914883ac53c9"
cas_config_opid_index: -1
I20260811 06:56:33.326846 15499 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } }
I20260811 06:56:33.331141 16364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:33.330708 16460 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } }
I20260811 06:56:33.334908 16460 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.40.61:40483
I20260811 06:56:33.335654 16460 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } }
I20260811 06:56:33.335897 16460 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:33.335947 16287 master.cc:537] Master@127.4.40.61:40483 shutting down...
I20260811 06:56:33.347656 16364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Refusing update from remote peer d0c6a3f5057249c1b2cf914883ac53c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 06:56:33.348891 16460 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:33.352299 16287 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 06:56:33.352411 15462 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 06:56:33.352843 16287 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:33.353210 16287 tablet_replica.cc:330] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: stopping tablet replica
W20260811 06:56:33.355263 15462 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 -> Peer f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61:40483): Couldn't send request to peer f45dd7db5d6444a998c5afd3ede4fdc2. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.60.keytab.
Entry for principal oryx/127.4.40.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.60.keytab.
I20260811 06:56:33.363049 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:37825
--webserver_interface=127.4.40.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.60.keytab
--principal=oryx/127.4.40.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:56:33.371491 16287 master.cc:552] Master@127.4.40.61:40483 shutdown complete.
I20260811 06:56:33.371639 16287 master_runner.cc:305] Clearing existing system tablet
I20260811 06:56:33.372152 16287 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:33.372252 16287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:33.375391 16287 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 06:56:33.377168 16471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:33.377517 16287 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:33.377866 16287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:56:33.377988 16287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "9e118dd57d6cda60ccbd46762afd6d77"
server_key_iv: "d5037bcc6d5ab0c7df44078dcffa2c0e"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.383150 16287 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:33.390404 16287 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:56:33.390637 16287 log.cc:1196] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 06:56:33.391160 16287 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Deleting consensus metadata
I20260811 06:56:33.391485 16287 master_runner.cc:315] Copying system tablet from 127.4.40.62:45025
I20260811 06:56:33.394162 16287 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: tablet copy: Beginning tablet copy session from remote peer at address 127.4.40.62:45025
I20260811 06:56:33.415458 16479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.394923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41034)
0811 06:56:33.400525 (+ 5602us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.400550 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.400718 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.401139 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.401145 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.401711 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.402012 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.402025 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.403581 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.403587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.405199 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.405215 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.406127 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.406133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.406318 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.407029 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.407053 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.408195 (+ 1142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.412200 (+ 4005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.412206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.412210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.412731 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.413217 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.413222 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.413225 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.413415 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.414249 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.414255 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.414434 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.414600 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.414954 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5451,"thread_start_us":5317,"threads_started":1}
I20260811 06:56:33.416492 16417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.395054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41034 (local address 127.4.40.62:45025)
0811 06:56:33.395293 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.395299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.400667 (+ 5368us) server_negotiation.cc:407] Connection header received
0811 06:56:33.400869 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.400875 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.400997 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.401105 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.402246 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.403431 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.405364 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.405954 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.406091 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.408417 (+ 2326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.408424 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.408428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.408468 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.412041 (+ 3573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.412892 (+ 851us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.412898 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.412902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.413081 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.413557 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.413562 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.413566 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.414032 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.414185 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.415765 (+ 1580us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.416007 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 06:56:33.417352 15531 tablet_copy_service.cc:140] P d0c6a3f5057249c1b2cf914883ac53c9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f45dd7db5d6444a998c5afd3ede4fdc2 ({username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:41034)
I20260811 06:56:33.417680 15531 tablet_copy_service.cc:161] P d0c6a3f5057249c1b2cf914883ac53c9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f45dd7db5d6444a998c5afd3ede4fdc2 at {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:41034: session id = f45dd7db5d6444a998c5afd3ede4fdc2-00000000000000000000000000000000
I20260811 06:56:33.422433 15531 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:56:33.424837 16287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:56:33.431777 16287 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: tablet copy: Starting download of 0 data blocks...
I20260811 06:56:33.432152 16287 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: tablet copy: Starting download of 1 WAL segments...
I20260811 06:56:33.434971 16287 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:56:33.439348 15531 tablet_copy_service.cc:342] P d0c6a3f5057249c1b2cf914883ac53c9: Request end of tablet copy session f45dd7db5d6444a998c5afd3ede4fdc2-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:41034
I20260811 06:56:33.439566 15531 tablet_copy_service.cc:433] P d0c6a3f5057249c1b2cf914883ac53c9: ending tablet copy session f45dd7db5d6444a998c5afd3ede4fdc2-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f45dd7db5d6444a998c5afd3ede4fdc2
I20260811 06:56:33.453645 16287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:33.457515 16484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:33.460516 16481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:33.468879 16488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:33.469373 16287 server_base.cc:1034] running on GCE node
I20260811 06:56:33.469738 16287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:33.470451 16287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:33.471673 16287 hybrid_clock.cc:648] HybridClock initialized: now 1786431393471620 us; error 35 us; skew 500 ppm
I20260811 06:56:33.471910 16287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:33.473836 16287 init.cc:377] Logged in from keytab as oryx/127.4.40.61@KRBTEST.COM (short username oryx)
I20260811 06:56:33.475476 16287 webserver.cc:460] Webserver started at http://127.4.40.61:33327/ using document root <none> and password file <none>
I20260811 06:56:33.475811 16287 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:33.475910 16287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:33.478370 16287 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:56:33.480078 16496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:33.480556 16287 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 06:56:33.480715 16287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "9e118dd57d6cda60ccbd46762afd6d77"
server_key_iv: "d5037bcc6d5ab0c7df44078dcffa2c0e"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.480880 16287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:33.490381 16287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:33.493832 16287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:33.525090 16287 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.61:40483
I20260811 06:56:33.525112 16595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.61:40483 every 8 connection(s)
I20260811 06:56:33.526057 16287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
I20260811 06:56:33.533133 16596 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:56:33.535797 16596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Bootstrap starting.
I20260811 06:56:33.549191 16596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:56:33.549685 16596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Bootstrap complete.
I20260811 06:56:33.550541 16596 raft_consensus.cc:348] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } }
I20260811 06:56:33.550781 16596 raft_consensus.cc:729] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f45dd7db5d6444a998c5afd3ede4fdc2, State: Initialized, Role: LEARNER
I20260811 06:56:33.551035 16596 consensus_queue.cc:260] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } }
I20260811 06:56:33.551921 16599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } }
I20260811 06:56:33.552275 16596 sys_catalog.cc:564] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:33.552395 16599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:56:33.563457 16615 catalog_manager.cc:1259] Loaded cluster ID: 1d9658236d6049c2b98b8031f9e4b061
I20260811 06:56:33.563587 16615 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: loading cluster ID for follower catalog manager: success
I20260811 06:56:33.566025 16615 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: acquiring CA information for follower catalog manager: success
I20260811 06:56:33.567936 16615 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:56:33.671752 16280 heartbeater.cc:503] Master 127.4.40.62:45025 was elected leader, sending a full tablet report...
W20260811 06:56:33.808736 16464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:33.809036 16464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:33.809130 16464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:33.817001 16464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:33.817102 16464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:33.817148 16464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:33.817206 16464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:33.817251 16464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:33.829592 16464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.60.keytab
--principal=oryx/127.4.40.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:37825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.40.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:33.830806 16464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:33.832216 16464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:33.844300 16631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:33.844396 16624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:33.844374 16627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:33.845412 16464 server_base.cc:1034] running on GCE node
I20260811 06:56:33.846197 16464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:33.847522 16464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:33.848754 16464 hybrid_clock.cc:648] HybridClock initialized: now 1786431393848716 us; error 51 us; skew 500 ppm
I20260811 06:56:33.849102 16464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:33.852299 16464 init.cc:377] Logged in from keytab as oryx/127.4.40.60@KRBTEST.COM (short username oryx)
I20260811 06:56:33.853837 16464 webserver.cc:460] Webserver started at http://127.4.40.60:44485/ using document root <none> and password file <none>
I20260811 06:56:33.854504 16464 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:33.854604 16464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:33.854933 16464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:33.858071 16464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/instance:
uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "961a22642f508681ccd9e4d9ed1f0b9b"
server_key_iv: "46d65a008b30db81f76f4e519b2802e3"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.858783 16464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal/instance:
uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "961a22642f508681ccd9e4d9ed1f0b9b"
server_key_iv: "46d65a008b30db81f76f4e519b2802e3"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.863783 16464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:56:33.867071 16642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:33.868458 16464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:56:33.868638 16464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "961a22642f508681ccd9e4d9ed1f0b9b"
server_key_iv: "46d65a008b30db81f76f4e519b2802e3"
server_key_version: "encryptionkey@0"
I20260811 06:56:33.868829 16464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:33.883406 16464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:33.887571 16464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:33.887856 16464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:33.907626 16746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.60:37825 every 8 connection(s)
I20260811 06:56:33.907624 16464 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.60:37825
I20260811 06:56:33.909211 16464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
I20260811 06:56:33.910179 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.902130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45240 (local address 127.4.40.61:40483)
0811 06:56:33.903370 (+ 1240us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.903375 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.903390 (+ 15us) server_negotiation.cc:407] Connection header received
0811 06:56:33.903453 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.903457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.903589 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.903731 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:33.904351 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.905772 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.908304 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.909075 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.909199 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.909412 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.909678 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.909776 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1122,"thread_start_us":1008,"threads_started":1}
I20260811 06:56:33.910228 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.901976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45240)
0811 06:56:33.902614 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.902639 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.902813 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.903808 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.903816 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.903859 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:33.904144 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.904162 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.905927 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.905934 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.908147 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.908163 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.909280 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.909287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.909472 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.909536 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.909652 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.909824 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":255,"threads_started":1}
I20260811 06:56:33.911505 16554 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 06:56:33.912061 16599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c6a3f5057249c1b2cf914883ac53c9. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } }
I20260811 06:56:33.912272 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 16464
I20260811 06:56:33.912375 16599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:56:33.914162 16748 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:33.922415 16748 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.60@KRBTEST.COM
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.60@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:33.935894 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.914984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.4.40.60:37825)
0811 06:56:33.915841 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.915852 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.915910 (+ 58us) server_negotiation.cc:407] Connection header received
0811 06:56:33.917691 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.917711 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.918081 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.918468 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.919985 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.921334 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.922631 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.923338 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.923543 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.928485 (+ 4942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.928492 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.928502 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.928545 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.931792 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.932578 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.932588 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.932600 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.932823 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.933229 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.933234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.933238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.933792 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.934083 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.934552 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.934954 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":354,"threads_started":1}
I20260811 06:56:33.939169 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.924897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.4.40.62:45025)
0811 06:56:33.925986 (+ 1089us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.925991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.926005 (+ 14us) server_negotiation.cc:407] Connection header received
0811 06:56:33.926047 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.926051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.926104 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.926232 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.927308 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.928142 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.929564 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.929977 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.930120 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.933367 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.933372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.933375 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.933416 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.936295 (+ 2879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.937084 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.937092 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.937095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.937234 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.937632 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.937637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.937641 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.937907 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.938069 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.938541 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.938770 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":986,"thread_start_us":906,"threads_started":1}
I20260811 06:56:33.939201 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.924761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41042)
0811 06:56:33.925479 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.925520 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.925654 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.926295 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.926303 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.926901 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.927150 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.927159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.928293 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.928305 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.929405 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.929418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.930169 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.930175 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.930344 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.930967 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.930984 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.933182 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.936419 (+ 3237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.936436 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.936443 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.936954 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.937349 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.937354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.937358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.937511 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.938045 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.938053 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.938220 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.938422 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.938787 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":329,"threads_started":1}
I20260811 06:56:33.940649 16748 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.60@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
I20260811 06:56:33.952745 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.941382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45254)
0811 06:56:33.941589 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.941604 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.941678 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.941927 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.941930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.942217 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.942400 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.942410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.943406 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.943409 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.944364 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.944376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.945142 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.945149 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.945331 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.945808 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.945824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.947666 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.950375 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.950380 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.950383 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.950747 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.951126 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.951131 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.951134 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.951266 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.951778 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.951782 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.951944 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.952082 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.952394 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:56:33.952765 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.941411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.4.40.61:40483)
0811 06:56:33.941630 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.941634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.941651 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:56:33.941756 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.941760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.941817 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.941894 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.942530 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.943244 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.944576 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.944965 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.945109 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.947857 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.947861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.947864 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.947896 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.950238 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.950866 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.950872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.950875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.951001 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.951375 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.951380 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.951383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.951631 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.951813 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.952213 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.952436 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:56:33.953958 16748 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:56:33 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431393, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.60@KRBTEST.COM for oryx/127.4.40.60@KRBTEST.COM
I20260811 06:56:33.966565 16754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.954734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.0.0.1:33252)
0811 06:56:33.954901 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:33.954915 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:33.955065 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:33.955335 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:33.955339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:33.955756 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:33.955883 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.955891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.956867 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.956875 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.957770 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:33.957782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.958490 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.958496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:33.958682 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.959221 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:33.959236 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:33.961307 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:33.964202 (+ 2895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.964207 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.964210 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.964555 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.964925 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:33.964930 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:33.964933 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:33.965108 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:33.965598 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:33.965602 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:33.965731 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 06:56:33.965867 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 06:56:33.966135 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:56:33.966663 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:33.954909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.4.40.60:37825)
0811 06:56:33.955079 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:33.955084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:33.955103 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:56:33.955158 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:33.955175 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:33.955229 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:33.955319 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:33.956019 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.956744 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.957900 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:33.958307 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:33.958429 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:33.961495 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:33.961501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:33.961506 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:33.961538 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:33.964011 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.964668 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.964674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.964678 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.964805 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:33.965225 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:33.965230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:33.965233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:33.965482 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:33.965631 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:33.965951 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 06:56:33.966284 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":114,"server-negotiator.queue_time_us":70}
I20260811 06:56:33.971309 16748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Bootstrap starting.
I20260811 06:56:33.974594 16748 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:33.975698 16748 log.cc:824] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:33.978458 16748 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: No bootstrap required, opened a new log
I20260811 06:56:33.985857 16748 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } }
I20260811 06:56:33.986145 16748 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:33.986265 16748 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, State: Initialized, Role: FOLLOWER
I20260811 06:56:33.986817 16748 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } }
I20260811 06:56:33.987682 16757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } } }
I20260811 06:56:33.988035 16757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:56:33.988668 16748 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:56:33.998642 16775 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:56:33.998772 16775 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:56:34.002452 16464 master_runner.cc:428] Detected that this master 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 is joining an existing cluster
I20260811 06:56:34.002549 16464 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.40.60:37825
I20260811 06:56:34.022607 16757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:34.022985 16757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } }
I20260811 06:56:34.025784 16554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" is_pre_election: true
I20260811 06:56:34.025839 15513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" is_pre_election: true
I20260811 06:56:34.025655 16757 leader_election.cc:288] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61:40483)
I20260811 06:56:34.028052 16644 leader_election.cc:302] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1; no voters: d0c6a3f5057249c1b2cf914883ac53c9, f45dd7db5d6444a998c5afd3ede4fdc2
I20260811 06:56:34.029903 16757 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:56:34.041813 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 15595
I20260811 06:56:34.042536 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.006549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41046 (local address 127.4.40.62:45025)
0811 06:56:34.006745 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.006751 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.021032 (+ 14281us) server_negotiation.cc:407] Connection header received
0811 06:56:34.021705 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.021712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.021813 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.021911 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.025680 (+ 3769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.026763 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.031026 (+ 4263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.031522 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.031674 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.034965 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.034972 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.034976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.035027 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.038788 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.039925 (+ 1137us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.039931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.039935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.040139 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.040649 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.040654 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.040658 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.041041 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.041193 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.041836 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.042183 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:56:34.043632 16727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.008188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45264 (local address 127.4.40.61:40483)
0811 06:56:34.008379 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.008385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.022211 (+ 13826us) server_negotiation.cc:407] Connection header received
0811 06:56:34.022702 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.022710 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.022822 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.022897 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.028518 (+ 5621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.029194 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.031324 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.032013 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.032156 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.034915 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.034922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.034927 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.034965 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.038929 (+ 3964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.039974 (+ 1045us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.039981 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.039986 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.040321 (+ 335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.040858 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.040862 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.040865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.041204 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.041342 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.042003 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.042256 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:56:34.045436 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.006584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41046)
0811 06:56:34.020894 (+ 14310us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.020916 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.021530 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.022112 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.022119 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.023399 (+ 1280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.025044 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.025060 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.027399 (+ 2339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.027402 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.030878 (+ 3476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.030893 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.031946 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.031953 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.032222 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.033425 (+ 1203us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.033448 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.034744 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.038956 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.038961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.038966 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.039768 (+ 802us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.040303 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.040307 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.040310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.040506 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.041279 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.041284 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.041505 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.043595 (+ 2090us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.044308 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14128,"mutex_wait_us":66,"spinlock_wait_cycles":43264,"thread_start_us":15199,"threads_started":1}
I20260811 06:56:34.046043 16782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.007581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45264)
0811 06:56:34.022100 (+ 14519us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.022122 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.022516 (+ 394us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.023243 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.023327 (+ 84us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.027264 (+ 3937us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.028364 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.028380 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.029712 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.029718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.031186 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.031196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.032469 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.032483 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.032624 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.033253 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.033282 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.034702 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.039087 (+ 4385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.039095 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.039101 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.039827 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.040508 (+ 681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.040516 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.040521 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.040716 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.041431 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.041435 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.041682 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.044156 (+ 2474us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.044509 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14374,"mutex_wait_us":1,"thread_start_us":13473,"threads_started":1}
I20260811 06:56:34.051812 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.006883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.0.0.1:33254)
0811 06:56:34.021590 (+ 14707us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.021630 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.022091 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.023672 (+ 1581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.023676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.025271 (+ 1595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.031397 (+ 6126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.031408 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.033001 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.033009 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.034705 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.034718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.035971 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.035978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.036178 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.036801 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.036827 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.039053 (+ 2226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.048952 (+ 9899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.048973 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.048977 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.049481 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.049885 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.049889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.049893 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.050054 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.050755 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.050760 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.050916 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.051073 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.051352 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14577,"thread_start_us":14374,"threads_started":1}
I20260811 06:56:34.052749 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.007086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33254 (local address 127.4.40.60:37825)
0811 06:56:34.007277 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.007286 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.021716 (+ 14430us) server_negotiation.cc:407] Connection header received
0811 06:56:34.022215 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.022222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.022295 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.022359 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.031552 (+ 9193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.032740 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.034843 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.035575 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.035709 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.039247 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.039254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.039259 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.039298 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.047153 (+ 7855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.049610 (+ 2457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.049615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.049619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.049765 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.050180 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.050185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.050188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.050542 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.050705 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.052173 (+ 1468us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.052388 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":38}
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.1@KRBTEST.COM
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.62@KRBTEST.COM for oryx/127.4.40.60@KRBTEST.COM
I20260811 06:56:34.068493 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.043136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53844 (local address 127.4.40.1:46877)
0811 06:56:34.045251 (+ 2115us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.045260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.045291 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:56:34.045359 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.045367 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.045681 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.045852 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.047289 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.048855 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.050807 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.051706 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.058776 (+ 7070us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.059175 (+ 399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.059191 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.059215 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.059258 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.064744 (+ 5486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.065873 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.065886 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.065893 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.066101 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.066621 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.066625 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.066630 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.066993 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.067340 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.067711 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.067991 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1937,"thread_start_us":1743,"threads_started":1}
I20260811 06:56:34.069644 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.051244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.0.0.1:33258)
0811 06:56:34.051618 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.051637 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.051742 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.053372 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.053378 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.053843 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.054062 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.054076 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.056081 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.056087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.057493 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.057508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.058642 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.058648 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.058837 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.059470 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.059494 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.062182 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.066595 (+ 4413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.066601 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.066605 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.067155 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.067600 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.067605 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.067609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.067761 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.068463 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.068470 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.068858 (+ 388us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.069012 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.069283 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249}
I20260811 06:56:34.069660 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.051944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.4.40.60:37825)
0811 06:56:34.052957 (+ 1013us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.052974 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.052991 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:56:34.053083 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.053087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.053193 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.054232 (+ 1039us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.054685 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.055823 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.057682 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.058449 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.062363 (+ 3914us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.062539 (+ 176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.062544 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.062549 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.062581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.066424 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.067323 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.067328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.067331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.067464 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.067911 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.067921 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.067927 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.068291 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.068440 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.069110 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.069323 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":14608,"threads_started":1}
I20260811 06:56:34.070041 15793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:34.071312 15499 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.40.60:37825: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
member_type: NON_VOTER
last_known_addr {
host: "127.4.40.60"
port: 37825
}
attrs {
promote: true
}
}
dest_uuid: "d0c6a3f5057249c1b2cf914883ac53c9"
cas_config_opid_index: 5
I20260811 06:56:34.072376 15499 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } }
I20260811 06:56:34.074210 16717 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.076542 16554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 LEARNER]: Refusing update from remote peer d0c6a3f5057249c1b2cf914883ac53c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 06:56:34.077190 16792 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 06:56:34.080029 16706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:34.079691 16554 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.083351 16599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.083659 16599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:56:34.083650 16706 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Refusing update from remote peer d0c6a3f5057249c1b2cf914883ac53c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 06:56:34.084453 16717 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.40.60:37825
I20260811 06:56:34.085011 16464 master.cc:537] Master@127.4.40.60:37825 shutting down...
I20260811 06:56:34.092661 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.092978 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:34.101989 16799 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.010s
I20260811 06:56:34.103145 16717 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: attempting to promote NON_VOTER f45dd7db5d6444a998c5afd3ede4fdc2 to VOTER
I20260811 06:56:34.113890 16717 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } }
I20260811 06:56:34.116163 16554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 LEARNER]: Refusing update from remote peer d0c6a3f5057249c1b2cf914883ac53c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 06:56:34.116031 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: NON_VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: true } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.116425 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:34.117051 16794 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 06:56:34.119374 16792 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.122412 16792 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.122644 16792 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:34.122887 16799 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.123242 16799 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:34.123126 16554 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.127473 16464 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:34.127908 16464 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:34.128171 16464 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: stopping tablet replica
I20260811 06:56:34.129312 16599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.129609 16599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:56:34.138068 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.137555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.0.0.1:33268)
0811 06:56:34.137753 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.137884 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:37825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 06:56:34.138285 15463 consensus_peers.cc:487] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 -> Peer 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60:37825): Couldn't send request to peer 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1. Status: Network error: Client connection negotiation failed: client connection to 127.4.40.60:37825: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:56:34.148531 16464 master.cc:552] Master@127.4.40.60:37825 shutdown complete.
I20260811 06:56:34.148633 16464 master_runner.cc:305] Clearing existing system tablet
I20260811 06:56:34.149080 16464 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:34.149184 16464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:34.152084 16464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.001s
I20260811 06:56:34.153954 16805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:34.154402 16464 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:34.154654 16464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 06:56:34.154752 16464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "961a22642f508681ccd9e4d9ed1f0b9b"
server_key_iv: "46d65a008b30db81f76f4e519b2802e3"
server_key_version: "encryptionkey@0"
I20260811 06:56:34.159742 16464 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:34.167018 16464 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 06:56:34.167179 16464 log.cc:1196] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 06:56:34.167680 16464 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Deleting consensus metadata
I20260811 06:56:34.167939 16464 master_runner.cc:315] Copying system tablet from 127.4.40.62:45025
I20260811 06:56:34.171239 16464 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: tablet copy: Beginning tablet copy session from remote peer at address 127.4.40.62:45025
I20260811 06:56:34.195578 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.172107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41050)
0811 06:56:34.180639 (+ 8532us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.180692 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.180855 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.181320 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.181326 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.181940 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.182219 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.182244 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.183781 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.183788 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.185457 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.185473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.186486 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.186492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.186664 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.187433 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.187469 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.188654 (+ 1185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.192717 (+ 4063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.192728 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.192732 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.193219 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.193653 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.193658 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.193662 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.193842 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.194459 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.194464 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.194645 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.194791 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.195111 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8353,"thread_start_us":8185,"threads_started":1}
I20260811 06:56:34.195593 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.172195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41050 (local address 127.4.40.62:45025)
0811 06:56:34.172431 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.172438 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.180794 (+ 8356us) server_negotiation.cc:407] Connection header received
0811 06:56:34.181000 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.181006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.181109 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.181215 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.182488 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.183602 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.185638 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.186306 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.186446 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.188872 (+ 2426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.188878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.188883 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.188922 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.192592 (+ 3670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.193371 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.193377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.193381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.193542 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.193973 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.193978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.193982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.194323 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.194473 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.194942 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.195244 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 06:56:34.196720 15531 tablet_copy_service.cc:140] P d0c6a3f5057249c1b2cf914883ac53c9: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 ({username='oryx', principal='oryx/127.4.40.60@KRBTEST.COM'} at 127.0.0.1:41050)
I20260811 06:56:34.196894 15531 tablet_copy_service.cc:161] P d0c6a3f5057249c1b2cf914883ac53c9: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 at {username='oryx', principal='oryx/127.4.40.60@KRBTEST.COM'} at 127.0.0.1:41050: session id = 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1-00000000000000000000000000000000
I20260811 06:56:34.200392 15531 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 06:56:34.202361 16464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 06:56:34.209291 16464 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: tablet copy: Starting download of 0 data blocks...
I20260811 06:56:34.209613 16464 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: tablet copy: Starting download of 1 WAL segments...
I20260811 06:56:34.212419 16464 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 06:56:34.218479 15531 tablet_copy_service.cc:342] P d0c6a3f5057249c1b2cf914883ac53c9: Request end of tablet copy session 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.40.60@KRBTEST.COM'} at 127.0.0.1:41050
I20260811 06:56:34.218679 15531 tablet_copy_service.cc:433] P d0c6a3f5057249c1b2cf914883ac53c9: ending tablet copy session 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1
I20260811 06:56:34.220382 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 15595
I20260811 06:56:34.243240 16464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:34.246280 16819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:34.247407 16821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:34.254531 16827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:34.257478 16464 server_base.cc:1034] running on GCE node
I20260811 06:56:34.257819 16464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:34.258620 16464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:34.259827 16464 hybrid_clock.cc:648] HybridClock initialized: now 1786431394259782 us; error 34 us; skew 500 ppm
I20260811 06:56:34.260156 16464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:56:34.261782 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:46877
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:45025
--webserver_port=37679
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431394, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:34.262698 16464 init.cc:377] Logged in from keytab as oryx/127.4.40.60@KRBTEST.COM (short username oryx)
I20260811 06:56:34.265404 16464 webserver.cc:460] Webserver started at http://127.4.40.60:45013/ using document root <none> and password file <none>
I20260811 06:56:34.265758 16464 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:34.265875 16464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:34.269148 16464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:56:34.271184 16844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:34.271627 16464 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260811 06:56:34.271767 16464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
format_stamp: "Formatted at 2026-08-11 06:56:33 on dist-test-slave-7vjv"
server_key: "961a22642f508681ccd9e4d9ed1f0b9b"
server_key_iv: "46d65a008b30db81f76f4e519b2802e3"
server_key_version: "encryptionkey@0"
I20260811 06:56:34.271944 16464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:34.297253 16464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:34.301633 16464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:34.343014 16464 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.60:37825
I20260811 06:56:34.343075 16941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.60:37825 every 8 connection(s)
I20260811 06:56:34.344337 16464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
I20260811 06:56:34.350418 16944 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:56:34.355156 16944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Bootstrap starting.
I20260811 06:56:34.377235 16944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:56:34.377998 16944 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: Bootstrap complete.
I20260811 06:56:34.379357 16944 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } }
I20260811 06:56:34.379841 16944 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, State: Initialized, Role: LEARNER
I20260811 06:56:34.380216 16944 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } }
I20260811 06:56:34.381791 16944 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:34.381453 16950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.382015 16950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: This master's current role is: LEARNER
I20260811 06:56:34.391662 16971 catalog_manager.cc:1259] Loaded cluster ID: 1d9658236d6049c2b98b8031f9e4b061
I20260811 06:56:34.391808 16971 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: loading cluster ID for follower catalog manager: success
I20260811 06:56:34.395627 16971 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: acquiring CA information for follower catalog manager: success
I20260811 06:56:34.398056 16971 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 06:56:34.650529 16845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:34.650815 16845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:34.651000 16845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 06:56:34.652601 16978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.641674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.0.0.1:33276)
0811 06:56:34.646854 (+ 5180us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.646879 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.647072 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.647592 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.647596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.647626 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:34.647861 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.647869 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.649227 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.649233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.650459 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.650471 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.651760 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.651766 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.651941 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.651978 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.652129 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.652277 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5065,"thread_start_us":4974,"threads_started":1}
I20260811 06:56:34.652639 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.641856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33276 (local address 127.4.40.60:37825)
0811 06:56:34.647269 (+ 5413us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.647273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.647293 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:56:34.647353 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.647356 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.647427 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.647569 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:34.648033 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.649078 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.650622 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.651585 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.651729 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.651980 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.652192 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.652321 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5282,"thread_start_us":5113,"threads_started":1}
I20260811 06:56:34.653761 16892 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 06:56:34.654400 16950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0c6a3f5057249c1b2cf914883ac53c9. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:34.654762 16950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: This master's current role is: LEARNER
W20260811 06:56:34.659260 16845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:34.659413 16845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:34.659541 16845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:34.669605 16845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:46877
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=37679
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:34.671039 16845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:34.672593 16845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:34.686594 16985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:34.686687 16987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:34.687057 16984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:34.687580 16845 server_base.cc:1034] running on GCE node
I20260811 06:56:34.688395 16845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:34.689265 16845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:34.690485 16845 hybrid_clock.cc:648] HybridClock initialized: now 1786431394690444 us; error 28 us; skew 500 ppm
I20260811 06:56:34.690762 16845 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431394, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:34.694139 16845 init.cc:377] Logged in from keytab as oryx/127.4.40.1@KRBTEST.COM (short username oryx)
I20260811 06:56:34.695746 16845 webserver.cc:460] Webserver started at http://127.4.40.1:37679/ using document root <none> and password file <none>
I20260811 06:56:34.696574 16845 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:34.696707 16845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:34.702508 16845 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:56:34.705194 17002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:34.706482 16845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:34.706627 16845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "04bed8119bb44f1580b28e08aa5132ac"
format_stamp: "Formatted at 2026-08-11 06:56:31 on dist-test-slave-7vjv"
server_key: "56575d6f6cfbeabd917ab9a0a1c1882b"
server_key_iv: "47c0ba7bc423613821f18740d863d49c"
server_key_version: "encryptionkey@0"
I20260811 06:56:34.707566 16845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:34.722533 16845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:34.726740 16845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:34.726984 16845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:34.727845 16845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:34.729200 16845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:34.729287 16845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:34.729362 16845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:34.729420 16845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:34.763936 16845 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:46877
I20260811 06:56:34.764032 17203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:46877 every 8 connection(s)
I20260811 06:56:34.765394 16845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:34.772343 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 16845
I20260811 06:56:34.772790 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 15811
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431394, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.2@KRBTEST.COM
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431394, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.60@KRBTEST.COM
Aug 11 06:56:34 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431394, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:34.810077 16982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.777322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:49569 (local address 127.4.40.60:37825)
0811 06:56:34.777544 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.777550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.791794 (+ 14244us) server_negotiation.cc:407] Connection header received
0811 06:56:34.792311 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.792317 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.792409 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.792490 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.793731 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.794816 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.797053 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.797812 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.797923 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.802843 (+ 4920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.802849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.802856 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.802899 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.806759 (+ 3860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.807616 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.807621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.807624 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.807774 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.808253 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.808259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.808262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.808620 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.808740 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.809441 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.809691 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 06:56:34.810108 17216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.768266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:58921 (local address 127.4.40.61:40483)
0811 06:56:34.777628 (+ 9362us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.777636 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.783059 (+ 5423us) server_negotiation.cc:407] Connection header received
0811 06:56:34.787849 (+ 4790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.787855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.787941 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.788026 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.792468 (+ 4442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.793626 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.796105 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.796764 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.796873 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.801780 (+ 4907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.801788 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.801793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.801835 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.806182 (+ 4347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.807150 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.807155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.807159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.807312 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.807862 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.807868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.807870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.808281 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.808425 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.809478 (+ 1053us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.809691 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7363,"thread_start_us":7185,"threads_started":1}
I20260811 06:56:34.812177 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.768291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:58105 (local address 127.4.40.62:45025)
0811 06:56:34.775943 (+ 7652us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.775953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.790703 (+ 14750us) server_negotiation.cc:407] Connection header received
0811 06:56:34.790914 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.790919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.790994 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.791090 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.795860 (+ 4770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.797022 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.799010 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.799761 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.799910 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.804871 (+ 4961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.804878 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.804883 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.804935 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.808577 (+ 3642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.809479 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.809486 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.809489 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.809641 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.810122 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.810127 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.810129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.810487 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.810831 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.811456 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.811707 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7486,"thread_start_us":7434,"threads_started":1}
I20260811 06:56:34.812285 17226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.774040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.4.40.2:44375)
0811 06:56:34.787067 (+ 13027us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:34.787075 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:34.787102 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:56:34.787169 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:34.787177 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:34.787393 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:34.787538 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:34.790300 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.792148 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.794159 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.795010 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.798076 (+ 3066us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.802302 (+ 4226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:34.802316 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:34.802332 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:34.802371 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:34.807634 (+ 5263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.808576 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.808586 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.808589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.808759 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:34.809237 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:34.809245 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:34.809250 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:34.809718 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:34.810026 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:34.811430 (+ 1404us) server_negotiation.cc:299] Negotiation successful
0811 06:56:34.811875 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12829,"thread_start_us":12661,"threads_started":1}
I20260811 06:56:34.812434 17224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.768864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.4.40.1:58921)
0811 06:56:34.782882 (+ 14018us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.782934 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.786355 (+ 3421us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.788630 (+ 2275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.788641 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.789330 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.790074 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.790095 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.793834 (+ 3739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.793839 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.795890 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.795921 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.797269 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.797275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.797597 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.798407 (+ 810us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.798434 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.801519 (+ 3085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.806346 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.806357 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.806376 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.807003 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.807482 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.807487 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.807490 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.807723 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.808554 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.808562 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.808976 (+ 414us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.810337 (+ 1361us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.810922 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13648,"mutex_wait_us":45,"spinlock_wait_cycles":42240,"thread_start_us":23188,"threads_started":1}
I20260811 06:56:34.812937 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.776634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.4.40.1:49569)
0811 06:56:34.791683 (+ 15049us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.791704 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.792086 (+ 382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.792693 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.792699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.793236 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.793547 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.793558 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.795038 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.795046 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.796860 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.796876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.798259 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.798268 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.798520 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.799832 (+ 1312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.799859 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.802613 (+ 2754us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.806916 (+ 4303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.806921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.806924 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.807482 (+ 558us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.807920 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.807927 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.807929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.808119 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.808825 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.808829 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.808976 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.810651 (+ 1675us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.811038 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14891,"mutex_wait_us":133,"spinlock_wait_cycles":1280,"thread_start_us":15184,"threads_started":1}
I20260811 06:56:34.813738 17223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:34.768864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:58105)
0811 06:56:34.790607 (+ 21743us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:34.790625 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:34.790792 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:34.791289 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:34.791294 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:34.791950 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:34.792269 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.792286 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.797203 (+ 4917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.797212 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.798842 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:34.798857 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:34.800205 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:34.800213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:34.800353 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:34.801825 (+ 1472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:34.801845 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:34.804629 (+ 2784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:34.808733 (+ 4104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.808738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.808742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.809335 (+ 593us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.809800 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:34.809807 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:34.809811 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:34.809961 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:34.810921 (+ 960us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:34.810925 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:34.811137 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 06:56:34.812454 (+ 1317us) client_negotiation.cc:240] Negotiation successful
0811 06:56:34.812753 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21595,"mutex_wait_us":35,"spinlock_wait_cycles":22400,"thread_start_us":14060,"threads_started":1}
I20260811 06:56:34.813774 16031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:34.814190 17204 heartbeater.cc:347] Connected to a master server at 127.4.40.60:37825
I20260811 06:56:34.814590 17204 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:34.814661 17207 heartbeater.cc:347] Connected to a master server at 127.4.40.61:40483
I20260811 06:56:34.814797 17207 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:34.814931 17206 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45025
I20260811 06:56:34.815066 17206 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:34.815845 17207 heartbeater.cc:511] Master 127.4.40.61:40483 requested a full tablet report, sending...
I20260811 06:56:34.815881 17204 heartbeater.cc:511] Master 127.4.40.60:37825 requested a full tablet report, sending...
I20260811 06:56:34.815907 17206 heartbeater.cc:511] Master 127.4.40.62:45025 requested a full tablet report, sending...
I20260811 06:56:34.817919 15499 ts_manager.cc:194] Re-registered known tserver with Master: 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877)
I20260811 06:56:34.818364 16531 ts_manager.cc:194] Registered new tserver with Master: 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877)
I20260811 06:56:34.818679 16876 ts_manager.cc:194] Registered new tserver with Master: 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877)
I20260811 06:56:34.819020 15499 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:58105
I20260811 06:56:34.904665 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 15811
I20260811 06:56:34.931672 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:44375
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:45025
--webserver_port=36201
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:56:35.259379 16976 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: NON_VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: true } } }
I20260811 06:56:35.259753 16976 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:35.261066 17235 raft_consensus.cc:1050] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: attempting to promote NON_VOTER 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 to VOTER
I20260811 06:56:35.261729 17235 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:35.263707 16892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 LEARNER]: Refusing update from remote peer d0c6a3f5057249c1b2cf914883ac53c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:56:35.263773 16554 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Refusing update from remote peer d0c6a3f5057249c1b2cf914883ac53c9: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 06:56:35.264412 17235 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:56:35.264806 17236 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 06:56:35.268142 16976 raft_consensus.cc:2932] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.269696 16892 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.272533 16976 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.272775 16976 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:35.272413 16554 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.273052 17236 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.273115 17235 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.273435 17236 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:35.273492 17235 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:35.279455 17252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.279799 17252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:56:35.280228 17253 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:35.280625 17253 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 06:56:35.287909 17233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:35.288273 17233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:35.288436 17233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:35.296010 17233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:35.296172 17233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:35.296370 17233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:35.306613 17233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:44375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=36201
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:35.308013 17233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:35.309412 17233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:35.323451 17265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:35.323616 17261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:35.323771 17262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:35.324462 17233 server_base.cc:1034] running on GCE node
I20260811 06:56:35.325002 17233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:35.325848 17233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:35.327064 17233 hybrid_clock.cc:648] HybridClock initialized: now 1786431395327012 us; error 38 us; skew 500 ppm
I20260811 06:56:35.327390 17233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:35 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:35.330601 17233 init.cc:377] Logged in from keytab as oryx/127.4.40.2@KRBTEST.COM (short username oryx)
I20260811 06:56:35.332402 17233 webserver.cc:460] Webserver started at http://127.4.40.2:36201/ using document root <none> and password file <none>
I20260811 06:56:35.333109 17233 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:35.333285 17233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:35.338963 17233 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 06:56:35.341850 17272 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:35.343169 17233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 06:56:35.343351 17233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "e0cb6e18a37446a4ba804fb73c3077ac"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "27914e2b9e9dfffb75b56a3284b0219d"
server_key_iv: "b6a104d486420b5969531891e12dae24"
server_key_version: "encryptionkey@0"
I20260811 06:56:35.344354 17233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:35.390861 17233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:35.394834 17233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:35.395187 17233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:35.396265 17233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:35.397708 17233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:35.397835 17233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:35.397974 17233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:35.398083 17233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:35.435117 17233 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:44375
I20260811 06:56:35.435173 17483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:44375 every 8 connection(s)
I20260811 06:56:35.436707 17233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:35.440559 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 17233
I20260811 06:56:35.440886 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 16047
Aug 11 06:56:35 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.60@KRBTEST.COM
Aug 11 06:56:35 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
Aug 11 06:56:35 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
Aug 11 06:56:35 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431390, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.3@KRBTEST.COM
I20260811 06:56:35.478770 17491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.439488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:55559 (local address 127.4.40.62:45025)
0811 06:56:35.440050 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:35.440057 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:35.456521 (+ 16464us) server_negotiation.cc:407] Connection header received
0811 06:56:35.456724 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:35.456733 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:35.456848 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:35.456971 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:35.460071 (+ 3100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.461259 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.463587 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.464073 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.464267 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.470749 (+ 6482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:35.470754 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:35.470758 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:35.470802 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:35.474542 (+ 3740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.475823 (+ 1281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.475829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.475833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.475966 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.476490 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.476495 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.476498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.476840 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:35.476970 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:35.478116 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0811 06:56:35.478358 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":3739,"threads_started":1}
I20260811 06:56:35.479406 17496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.440440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:54875 (local address 127.4.40.60:37825)
0811 06:56:35.454592 (+ 14152us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:35.454596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:35.454620 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:56:35.455782 (+ 1162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:35.455791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:35.455912 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:35.456045 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:35.459371 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.460701 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.463541 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.464176 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.464294 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.469575 (+ 5281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:35.469583 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:35.469591 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:35.469639 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:35.474397 (+ 4758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.475882 (+ 1485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.475889 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.475894 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.476053 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.476629 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.476635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.476638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.476982 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:35.477148 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:35.478365 (+ 1217us) server_negotiation.cc:299] Negotiation successful
0811 06:56:35.478601 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13998,"thread_start_us":13818,"threads_started":1}
I20260811 06:56:35.479945 17497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.440236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:55559)
0811 06:56:35.456413 (+ 16177us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:35.456433 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:35.456567 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:35.457450 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:35.457456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:35.458185 (+ 729us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:35.459895 (+ 1710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:35.459910 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.461688 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.461693 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:35.463449 (+ 1756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:35.463471 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.464636 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.464641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:35.464971 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.465968 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:35.465988 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:35.470365 (+ 4377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:35.474675 (+ 4310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:35.474679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:35.474684 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:35.475722 (+ 1038us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:35.476080 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:35.476087 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:35.476120 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:35.476375 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:35.477224 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:35.477228 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:35.477674 (+ 446us) client_negotiation.cc:769] Sending connection context
0811 06:56:35.478004 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 06:56:35.478539 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15972,"mutex_wait_us":1555,"spinlock_wait_cycles":49536,"thread_start_us":15277,"threads_started":1}
I20260811 06:56:35.480739 17494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.440760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.4.40.2:55165)
0811 06:56:35.457142 (+ 16382us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:35.457161 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:35.457268 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:35.459434 (+ 2166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:35.459440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:35.461032 (+ 1592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:35.461489 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:35.461501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.463019 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.463026 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:35.465213 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:35.465229 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.466568 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.466581 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:35.466809 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.467511 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:35.467543 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:35.470439 (+ 2896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:35.474656 (+ 4217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:35.474669 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:35.474684 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:35.475722 (+ 1038us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:35.477101 (+ 1379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:35.477105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:35.477109 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:35.477344 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:35.479162 (+ 1818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:35.479170 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:35.479295 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 06:56:35.479444 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 06:56:35.479735 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16248,"thread_start_us":11374,"threads_started":1}
I20260811 06:56:35.481446 17493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.439909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:55165 (local address 127.4.40.61:40483)
0811 06:56:35.459116 (+ 19207us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:35.459123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:35.459141 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:56:35.459208 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:35.459214 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:35.459297 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:35.459420 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:35.461661 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.462806 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.465465 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.466090 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.466235 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.470628 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:35.470635 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:35.470641 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:35.470703 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:35.474484 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.476222 (+ 1738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.476227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.476230 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.476375 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.477486 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.477491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.477494 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.477859 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:35.479095 (+ 1236us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:35.480166 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0811 06:56:35.480436 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19096,"thread_start_us":18959,"threads_started":1}
I20260811 06:56:35.481590 17487 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45025
I20260811 06:56:35.481998 17487 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:35.482204 17489 heartbeater.cc:347] Connected to a master server at 127.4.40.61:40483
I20260811 06:56:35.482331 17489 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:35.483132 17501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.442979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36938 (local address 127.4.40.3:42639)
0811 06:56:35.457533 (+ 14554us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:35.457545 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:35.458402 (+ 857us) server_negotiation.cc:407] Connection header received
0811 06:56:35.458487 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:35.458495 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:35.458750 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:35.458916 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:35.461299 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.462812 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.464529 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.465416 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.465538 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.473708 (+ 8170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:35.473724 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:35.473739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:35.473783 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:35.477861 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.478817 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.478839 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.478844 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.479024 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:35.479538 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:35.479546 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:35.479551 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:35.480205 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:35.480524 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:35.482327 (+ 1803us) server_negotiation.cc:299] Negotiation successful
0811 06:56:35.482622 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14365,"thread_start_us":14191,"threads_started":1}
I20260811 06:56:35.483157 17487 heartbeater.cc:511] Master 127.4.40.62:45025 requested a full tablet report, sending...
I20260811 06:56:35.483621 17498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:35.440236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.4.40.2:54875)
0811 06:56:35.450578 (+ 10342us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:35.450641 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:35.455422 (+ 4781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:35.457190 (+ 1768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:35.457206 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:35.458185 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:35.459179 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:35.459200 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.461008 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.461015 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:35.463368 (+ 2353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:35.463382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:35.464592 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:35.464601 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:35.464973 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:35.465879 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:35.465922 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:35.469304 (+ 3382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:35.474684 (+ 5380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:35.474692 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:35.474698 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:35.475716 (+ 1018us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:35.476258 (+ 542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:35.476264 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:35.476275 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:35.476484 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:35.477187 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:35.477196 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:35.477734 (+ 538us) client_negotiation.cc:769] Sending connection context
0811 06:56:35.478004 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 06:56:35.480863 (+ 2859us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9928,"mutex_wait_us":2590,"spinlock_wait_cycles":8320,"thread_start_us":16409,"threads_started":1}
I20260811 06:56:35.484306 17489 heartbeater.cc:511] Master 127.4.40.61:40483 requested a full tablet report, sending...
I20260811 06:56:35.484578 16273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:35.485244 17485 heartbeater.cc:347] Connected to a master server at 127.4.40.60:37825
I20260811 06:56:35.485375 17485 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:35.485723 15499 ts_manager.cc:194] Re-registered known tserver with Master: e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375)
I20260811 06:56:35.485780 16531 ts_manager.cc:194] Registered new tserver with Master: e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375)
I20260811 06:56:35.485807 17485 heartbeater.cc:511] Master 127.4.40.60:37825 requested a full tablet report, sending...
I20260811 06:56:35.486881 15499 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:55559
I20260811 06:56:35.487062 16876 ts_manager.cc:194] Registered new tserver with Master: e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375)
I20260811 06:56:35.580511 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 16047
I20260811 06:56:35.610463 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:42639
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:45025
--webserver_port=42529
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:40145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:56:35.821349 17206 heartbeater.cc:503] Master 127.4.40.62:45025 was elected leader, sending a full tablet report...
W20260811 06:56:35.956979 17504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:35.957237 17504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:35.957405 17504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:35.964879 17504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:35.964979 17504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:35.965200 17504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:35.975042 17504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:40145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:42639
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=42529
--tserver_master_addrs=127.4.40.62:45025,127.4.40.61:40483,127.4.40.60:37825
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:35.976505 17504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:35.977844 17504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:35.991487 17515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:35.991437 17525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:35.991462 17517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:35.992735 17504 server_base.cc:1034] running on GCE node
I20260811 06:56:35.993326 17504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:35.994261 17504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:35.995464 17504 hybrid_clock.cc:648] HybridClock initialized: now 1786431395995425 us; error 31 us; skew 500 ppm
I20260811 06:56:35.995859 17504 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:35 dist-test-slave-7vjv krb5kdc[15418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:35.999109 17504 init.cc:377] Logged in from keytab as oryx/127.4.40.3@KRBTEST.COM (short username oryx)
I20260811 06:56:36.000682 17504 webserver.cc:460] Webserver started at http://127.4.40.3:42529/ using document root <none> and password file <none>
I20260811 06:56:36.001366 17504 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:36.001562 17504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:36.006897 17504 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:56:36.009721 17537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:36.011053 17504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 06:56:36.011225 17504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "78862879db8d48389ead036510fb4094"
format_stamp: "Formatted at 2026-08-11 06:56:32 on dist-test-slave-7vjv"
server_key: "a3171cc979f7fa1d9f928400afbeca57"
server_key_iv: "18d02d49c138a8637ee0154adb24a17f"
server_key_version: "encryptionkey@0"
I20260811 06:56:36.012235 17504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:36.037338 17504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:36.041225 17504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:36.041507 17504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:36.042454 17504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:36.043756 17504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:36.043840 17504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:36.043972 17504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:36.044070 17504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:36.079463 17504 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:42639
I20260811 06:56:36.079526 17745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:42639 every 8 connection(s)
I20260811 06:56:36.081087 17504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:36.088161 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 17504
Aug 11 06:56:36 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
Aug 11 06:56:36 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.61@KRBTEST.COM
Aug 11 06:56:36 dist-test-slave-7vjv krb5kdc[15418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431395, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.60@KRBTEST.COM
I20260811 06:56:36.129707 17761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.089605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:50035)
0811 06:56:36.100767 (+ 11162us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.100793 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.102872 (+ 2079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.103944 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.103955 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.105080 (+ 1125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:36.106201 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.106242 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.108129 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.108139 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.110863 (+ 2724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.110880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.112361 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.112370 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.112697 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.113523 (+ 826us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:36.113564 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:36.117550 (+ 3986us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:36.123648 (+ 6098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:36.123662 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:36.123683 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:36.124637 (+ 954us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:36.125185 (+ 548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:36.125192 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:36.125198 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:36.125481 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:36.126400 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:36.126411 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:36.126881 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.127252 (+ 371us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.127922 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10937,"mutex_wait_us":395,"thread_start_us":10282,"threads_started":1}
I20260811 06:56:36.131309 17752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.084594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:50035 (local address 127.4.40.62:45025)
0811 06:56:36.088674 (+ 4080us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.088681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.100915 (+ 12234us) server_negotiation.cc:407] Connection header received
0811 06:56:36.103324 (+ 2409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.103330 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.103445 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.103558 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.106495 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.107860 (+ 1365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.111061 (+ 3201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.111865 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.111976 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.117849 (+ 5873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.117859 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.117868 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.117922 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.123422 (+ 5500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.124817 (+ 1395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.124827 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.124833 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.125018 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.125665 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.125673 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.125680 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.126138 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.126302 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.130466 (+ 4164us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.130756 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3904,"thread_start_us":3766,"threads_started":1}
I20260811 06:56:36.132831 17747 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45025
I20260811 06:56:36.133206 17747 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:36.134374 17747 heartbeater.cc:511] Master 127.4.40.62:45025 requested a full tablet report, sending...
I20260811 06:56:36.135912 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.090042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.4.40.3:37197)
0811 06:56:36.101566 (+ 11524us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.101581 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.102859 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.103944 (+ 1085us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.103955 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.105890 (+ 1935us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:36.106343 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.106359 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.110027 (+ 3668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.110034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.111834 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.111862 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.121941 (+ 10079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.121949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.122167 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.122858 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:36.122882 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:36.126205 (+ 3323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:36.131494 (+ 5289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:36.131502 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:36.131506 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:36.132247 (+ 741us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:36.132805 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:36.132818 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:36.132824 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:36.133066 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:36.134582 (+ 1516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:36.134589 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:36.134775 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.134941 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.135340 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11399,"mutex_wait_us":131,"thread_start_us":11059,"threads_started":1}
I20260811 06:56:36.136376 15499 ts_manager.cc:194] Re-registered known tserver with Master: 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:36.137356 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.084649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:37197 (local address 127.4.40.60:37825)
0811 06:56:36.089231 (+ 4582us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.089238 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.101688 (+ 12450us) server_negotiation.cc:407] Connection header received
0811 06:56:36.103216 (+ 1528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.103224 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.103338 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.103461 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.106507 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.107861 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.112049 (+ 4188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.112728 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.112882 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.126493 (+ 13611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.126503 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.126507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.126558 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.131271 (+ 4713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.132424 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.132432 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.132437 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.132647 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.133220 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.133228 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.133233 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.133722 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.133895 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.136425 (+ 2530us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.136746 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4453,"thread_start_us":5392,"threads_started":1}
I20260811 06:56:36.137949 15499 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:50035
I20260811 06:56:36.143653 17746 heartbeater.cc:347] Connected to a master server at 127.4.40.60:37825
I20260811 06:56:36.143795 17746 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:36.144284 17746 heartbeater.cc:511] Master 127.4.40.60:37825 requested a full tablet report, sending...
I20260811 06:56:36.145730 16876 ts_manager.cc:194] Registered new tserver with Master: 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:36.146530 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.084594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:46365 (local address 127.4.40.61:40483)
0811 06:56:36.088185 (+ 3591us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.088196 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.099389 (+ 11193us) server_negotiation.cc:407] Connection header received
0811 06:56:36.103434 (+ 4045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.103441 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.103560 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.103649 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.112467 (+ 8818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.113630 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.117870 (+ 4240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.118822 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.118968 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.138445 (+ 19477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.138455 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.138462 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.138526 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.143107 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.143919 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.143927 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.143931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.144248 (+ 317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.144723 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.144730 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.144734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.145122 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.145285 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.145861 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.146123 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3375,"thread_start_us":3948,"threads_started":1}
I20260811 06:56:36.147758 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.089605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.4.40.3:46365)
0811 06:56:36.099170 (+ 9565us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.099228 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.103259 (+ 4031us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.103942 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.103954 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.111062 (+ 7108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:36.111867 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.111882 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.113915 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.113927 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.117656 (+ 3729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.117677 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.119810 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.119819 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.120079 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.120826 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:36.120856 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:36.138132 (+ 17276us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:36.143272 (+ 5140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:36.143277 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:36.143280 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:36.143769 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:36.144393 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:36.144400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:36.144406 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:36.144591 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:36.145375 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:36.145382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:36.145530 (+ 148us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.146822 (+ 1292us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.147191 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9261,"thread_start_us":9007,"threads_started":1}
I20260811 06:56:36.149892 17748 heartbeater.cc:347] Connected to a master server at 127.4.40.61:40483
I20260811 06:56:36.150028 17748 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:36.150394 17748 heartbeater.cc:511] Master 127.4.40.61:40483 requested a full tablet report, sending...
I20260811 06:56:36.151645 16531 ts_manager.cc:194] Registered new tserver with Master: 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:36.162634 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.113281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.4.40.60:37825)
0811 06:56:36.137648 (+ 24367us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.137657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.144213 (+ 6556us) server_negotiation.cc:407] Connection header received
0811 06:56:36.144267 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.144272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.144366 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.144448 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.147446 (+ 2998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.149016 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.150920 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.152644 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.152782 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.155703 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.155713 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.155718 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.155753 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.159087 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.160155 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.160160 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.160164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.160326 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.160758 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.160763 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.160779 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.161155 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.161393 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.162019 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.162262 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24230,"thread_start_us":28686,"threads_started":1}
I20260811 06:56:36.163019 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.104862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45274 (local address 127.4.40.61:40483)
0811 06:56:36.116213 (+ 11351us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.116223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.143660 (+ 27437us) server_negotiation.cc:407] Connection header received
0811 06:56:36.143720 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.143725 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.143799 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.143914 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.147497 (+ 3583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.148889 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.150580 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.151237 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.151385 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.155582 (+ 4197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.155591 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.155598 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.155646 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.159368 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.160350 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.160356 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.160360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.160554 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.161026 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.161031 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.161035 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.161505 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.161785 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.162257 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.162517 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11140,"thread_start_us":11474,"threads_started":1}
I20260811 06:56:36.164639 17768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.105069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41064 (local address 127.4.40.62:45025)
0811 06:56:36.115677 (+ 10608us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.115686 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.142935 (+ 27249us) server_negotiation.cc:407] Connection header received
0811 06:56:36.143251 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.143260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.143369 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.143452 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.148674 (+ 5222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.149742 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.152579 (+ 2837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.153367 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.153521 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.157465 (+ 3944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.157476 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.157483 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.157528 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.161017 (+ 3489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.162004 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.162010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.162014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.162205 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.162702 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.162707 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.162710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.163093 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.163536 (+ 443us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.163932 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.164183 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10444,"thread_start_us":8282,"threads_started":1}
I20260811 06:56:36.169299 15499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:56:36.176479 15499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:36.208382 17787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.198147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:53858)
0811 06:56:36.199053 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.199076 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.199214 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.200742 (+ 1528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.200750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.200816 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:36.201397 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.201414 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.202826 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.202834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.205626 (+ 2792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.205644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.207484 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.207489 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.207668 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.207707 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.207823 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.207958 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":368,"threads_started":1}
I20260811 06:56:36.209429 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.198401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53858 (local address 127.4.40.1:46877)
0811 06:56:36.200048 (+ 1647us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.200057 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.200115 (+ 58us) server_negotiation.cc:407] Connection header received
0811 06:56:36.200227 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.200235 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.200525 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.200756 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:36.201611 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.202691 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.205802 (+ 3111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.207305 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.207447 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.207675 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.207889 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.208044 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1452,"thread_start_us":1931,"threads_started":1}
I20260811 06:56:36.213167 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.199897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44375 (local address 127.0.0.1:33174)
0811 06:56:36.205478 (+ 5581us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.205495 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.205583 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.206154 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.206158 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.206184 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:36.206565 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.206574 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.208299 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.208304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.210416 (+ 2112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.210430 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.212240 (+ 1810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.212247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.212440 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.212491 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.212592 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.212729 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5493,"mutex_wait_us":77,"thread_start_us":4391,"threads_started":1}
I20260811 06:56:36.213306 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.200190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33174 (local address 127.4.40.2:44375)
0811 06:56:36.201798 (+ 1608us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.201809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.205603 (+ 3794us) server_negotiation.cc:407] Connection header received
0811 06:56:36.205723 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.205733 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.205961 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.206166 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:36.206730 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.208138 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.210626 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.211997 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.212178 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.212368 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.212808 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.213001 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":1059,"threads_started":1}
I20260811 06:56:36.213637 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.198886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42639 (local address 127.0.0.1:36946)
0811 06:56:36.204884 (+ 5998us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.204904 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.205044 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.206539 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.206548 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.206575 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:36.206948 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.206959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.208943 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.208949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.210954 (+ 2005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.210969 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.212573 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.212580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.212774 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.212805 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.212904 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.213050 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5585,"mutex_wait_us":1,"thread_start_us":6379,"threads_started":1}
I20260811 06:56:36.214090 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.199353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36946 (local address 127.4.40.3:42639)
0811 06:56:36.200140 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.200147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.205692 (+ 5545us) server_negotiation.cc:407] Connection header received
0811 06:56:36.205831 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.205839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.206070 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.206262 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:36.207169 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.208811 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.211114 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.212360 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.212491 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.212696 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.213592 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.213755 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":654,"threads_started":1}
I20260811 06:56:36.213855 17088 tablet_service.cc:1467] Processing CreateTablet for tablet e22da709254f4adcbf55c9d5aabc0a5c (DEFAULT_TABLE table=test-table [id=5b9ca482fac84c0691e314829339bbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:36.216082 17088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22da709254f4adcbf55c9d5aabc0a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:36.216970 17362 tablet_service.cc:1467] Processing CreateTablet for tablet e22da709254f4adcbf55c9d5aabc0a5c (DEFAULT_TABLE table=test-table [id=5b9ca482fac84c0691e314829339bbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:36.218043 17362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22da709254f4adcbf55c9d5aabc0a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:36.217625 17623 tablet_service.cc:1467] Processing CreateTablet for tablet e22da709254f4adcbf55c9d5aabc0a5c (DEFAULT_TABLE table=test-table [id=5b9ca482fac84c0691e314829339bbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:36.218990 17623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e22da709254f4adcbf55c9d5aabc0a5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:36.229882 17800 tablet_bootstrap.cc:492] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Bootstrap starting.
I20260811 06:56:36.232810 17802 tablet_bootstrap.cc:492] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Bootstrap starting.
I20260811 06:56:36.233243 17800 tablet_bootstrap.cc:654] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:36.234689 17800 log.cc:824] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:36.235271 17804 tablet_bootstrap.cc:492] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Bootstrap starting.
I20260811 06:56:36.236641 17802 tablet_bootstrap.cc:654] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:36.237936 17802 log.cc:824] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:36.238842 17804 tablet_bootstrap.cc:654] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:36.239512 17800 tablet_bootstrap.cc:492] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: No bootstrap required, opened a new log
I20260811 06:56:36.239820 17800 ts_tablet_manager.cc:1397] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260811 06:56:36.240020 17804 log.cc:824] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:36.241030 17802 tablet_bootstrap.cc:492] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: No bootstrap required, opened a new log
I20260811 06:56:36.241298 17802 ts_tablet_manager.cc:1397] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 06:56:36.242444 17804 tablet_bootstrap.cc:492] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: No bootstrap required, opened a new log
I20260811 06:56:36.242707 17804 ts_tablet_manager.cc:1397] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 06:56:36.250200 17804 raft_consensus.cc:348] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.250610 17804 raft_consensus.cc:374] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:36.250707 17804 raft_consensus.cc:729] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78862879db8d48389ead036510fb4094, State: Initialized, Role: FOLLOWER
I20260811 06:56:36.250545 17800 raft_consensus.cc:348] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.250952 17800 raft_consensus.cc:374] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:36.251065 17800 raft_consensus.cc:729] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cb6e18a37446a4ba804fb73c3077ac, State: Initialized, Role: FOLLOWER
I20260811 06:56:36.251447 17804 consensus_queue.cc:260] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.251657 17800 consensus_queue.cc:260] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.251816 17802 raft_consensus.cc:348] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.252275 17802 raft_consensus.cc:374] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:36.252409 17802 raft_consensus.cc:729] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bed8119bb44f1580b28e08aa5132ac, State: Initialized, Role: FOLLOWER
I20260811 06:56:36.253445 17804 ts_tablet_manager.cc:1428] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Time spent starting tablet: real 0.011s user 0.006s sys 0.005s
I20260811 06:56:36.253249 17802 consensus_queue.cc:260] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.253824 17747 heartbeater.cc:503] Master 127.4.40.62:45025 was elected leader, sending a full tablet report...
I20260811 06:56:36.255169 17800 ts_tablet_manager.cc:1428] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Time spent starting tablet: real 0.015s user 0.011s sys 0.006s
I20260811 06:56:36.255939 17487 heartbeater.cc:503] Master 127.4.40.62:45025 was elected leader, sending a full tablet report...
I20260811 06:56:36.263262 17810 raft_consensus.cc:482] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:36.263679 17810 raft_consensus.cc:504] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.265435 17802 ts_tablet_manager.cc:1428] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Time spent starting tablet: real 0.024s user 0.013s sys 0.008s
I20260811 06:56:36.267333 17810 leader_election.cc:288] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877), e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375)
I20260811 06:56:36.274757 17790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.267781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:54837 (local address 127.4.40.1:46877)
0811 06:56:36.267986 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.267994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.268335 (+ 341us) server_negotiation.cc:407] Connection header received
0811 06:56:36.268412 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.268416 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.268482 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.268575 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:36.269155 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.270145 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.272526 (+ 2381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.273746 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.273877 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.274000 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.274375 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.274495 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:56:36.274811 17760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.267849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.4.40.3:54837)
0811 06:56:36.268129 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.268146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.268227 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.268728 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.268753 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.268784 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:36.268967 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.268980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.270298 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.270302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.272315 (+ 2013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.272331 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.273892 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.273896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.274146 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.274241 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.274347 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.274501 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":43}
I20260811 06:56:36.274833 17758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.267578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44375 (local address 127.4.40.3:49185)
0811 06:56:36.267946 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:36.267968 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:36.268114 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:36.269191 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:36.269196 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:36.269245 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:36.269439 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.269452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.270652 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.270657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.272532 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:36.272543 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.274143 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.274149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:36.274371 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.274400 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:56:36.274487 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:56:36.274591 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":1}
I20260811 06:56:36.275370 17798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.268377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:49185 (local address 127.4.40.2:44375)
0811 06:56:36.268554 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.268561 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.268577 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:56:36.268630 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.268633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.268695 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.268785 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:36.269594 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.270518 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.272673 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.273924 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.274098 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.274167 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.274515 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.274646 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 06:56:36.275852 17208 tablet.cc:2369] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:36.275880 17124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22da709254f4adcbf55c9d5aabc0a5c" candidate_uuid: "78862879db8d48389ead036510fb4094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bed8119bb44f1580b28e08aa5132ac" is_pre_election: true
I20260811 06:56:36.276355 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22da709254f4adcbf55c9d5aabc0a5c" candidate_uuid: "78862879db8d48389ead036510fb4094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" is_pre_election: true
I20260811 06:56:36.276556 17124 raft_consensus.cc:2454] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78862879db8d48389ead036510fb4094 in term 0.
I20260811 06:56:36.276885 17401 raft_consensus.cc:2454] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78862879db8d48389ead036510fb4094 in term 0.
I20260811 06:56:36.277501 17543 leader_election.cc:302] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bed8119bb44f1580b28e08aa5132ac, 78862879db8d48389ead036510fb4094; no voters:
I20260811 06:56:36.277949 17810 raft_consensus.cc:2781] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:36.278113 17810 raft_consensus.cc:482] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:36.278227 17810 raft_consensus.cc:3037] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:36.280467 17810 raft_consensus.cc:504] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.281117 17810 leader_election.cc:288] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [CANDIDATE]: Term 1 election: Requested vote from peers 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877), e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375)
I20260811 06:56:36.281450 17124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22da709254f4adcbf55c9d5aabc0a5c" candidate_uuid: "78862879db8d48389ead036510fb4094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bed8119bb44f1580b28e08aa5132ac"
I20260811 06:56:36.281653 17124 raft_consensus.cc:3037] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:36.281603 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e22da709254f4adcbf55c9d5aabc0a5c" candidate_uuid: "78862879db8d48389ead036510fb4094" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cb6e18a37446a4ba804fb73c3077ac"
I20260811 06:56:36.281836 17401 raft_consensus.cc:3037] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:36.284039 17124 raft_consensus.cc:2454] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78862879db8d48389ead036510fb4094 in term 1.
I20260811 06:56:36.284457 17543 leader_election.cc:302] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bed8119bb44f1580b28e08aa5132ac, 78862879db8d48389ead036510fb4094; no voters:
I20260811 06:56:36.284531 17401 raft_consensus.cc:2454] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78862879db8d48389ead036510fb4094 in term 1.
I20260811 06:56:36.284773 17810 raft_consensus.cc:2781] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:36.284991 17810 raft_consensus.cc:686] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 1 LEADER]: Becoming Leader. State: Replica: 78862879db8d48389ead036510fb4094, State: Running, Role: LEADER
I20260811 06:56:36.285461 17810 consensus_queue.cc:237] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } }
I20260811 06:56:36.290951 15499 catalog_manager.cc:5495] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78862879db8d48389ead036510fb4094 (127.4.40.3). New cstate: current_term: 1 leader_uuid: "78862879db8d48389ead036510fb4094" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } health_report { overall_health: UNKNOWN } } }
I20260811 06:56:36.309685 17795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.304499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36956 (local address 127.4.40.3:42639)
0811 06:56:36.304728 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.304734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.304812 (+ 78us) server_negotiation.cc:407] Connection header received
0811 06:56:36.304994 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.304998 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.305085 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.305222 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:36.305533 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.306255 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.307537 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.307925 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.308068 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.308619 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:36.309103 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:36.309275 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.309329 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.309406 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:56:36.317837 17401 raft_consensus.cc:1261] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Refusing update from remote peer 78862879db8d48389ead036510fb4094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:36.317836 17124 raft_consensus.cc:1261] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Refusing update from remote peer 78862879db8d48389ead036510fb4094: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:36.318785 17810 consensus_queue.cc:1035] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:36.319161 17814 consensus_queue.cc:1035] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:36.376302 15499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:36.376602 15499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:36.384076 15499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d0c6a3f5057249c1b2cf914883ac53c9: Sending DeleteTablet for 3 replicas of tablet e22da709254f4adcbf55c9d5aabc0a5c
I20260811 06:56:36.385284 17088 tablet_service.cc:1514] Processing DeleteTablet for tablet e22da709254f4adcbf55c9d5aabc0a5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:36 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:53858
I20260811 06:56:36.385365 17623 tablet_service.cc:1514] Processing DeleteTablet for tablet e22da709254f4adcbf55c9d5aabc0a5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:36 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:36946
I20260811 06:56:36.385722 17362 tablet_service.cc:1514] Processing DeleteTablet for tablet e22da709254f4adcbf55c9d5aabc0a5c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:36 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:33174
I20260811 06:56:36.396502 17840 tablet_replica.cc:330] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: stopping tablet replica
I20260811 06:56:36.396718 17841 tablet_replica.cc:330] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: stopping tablet replica
I20260811 06:56:36.397318 17840 raft_consensus.cc:2229] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:36.397964 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 15434
I20260811 06:56:36.397955 17841 raft_consensus.cc:2229] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:36.398849 17841 raft_consensus.cc:2258] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:36.398972 15543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:36.399546 17840 raft_consensus.cc:2258] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:36.402355 17841 ts_tablet_manager.cc:1905] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:36.409539 17840 ts_tablet_manager.cc:1905] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:36.412721 17842 tablet_replica.cc:330] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: stopping tablet replica
I20260811 06:56:36.413262 17842 raft_consensus.cc:2229] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:36.414397 17842 raft_consensus.cc:2258] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:36.419668 17842 ts_tablet_manager.cc:1905] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:36.421574 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.388079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45282 (local address 127.4.40.61:40483)
0811 06:56:36.388336 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.388345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.396942 (+ 8597us) server_negotiation.cc:407] Connection header received
0811 06:56:36.398023 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.398030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.398124 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.398204 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.403569 (+ 5365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.404678 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.408372 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.409039 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.409141 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.414086 (+ 4945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.414092 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.414096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.414137 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.417857 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.419004 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.419011 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.419015 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.419179 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.419708 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.419713 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.419716 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.420082 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.420361 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.420982 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.421241 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 06:56:36.425941 17840 ts_tablet_manager.cc:1918] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:36.431409 17842 ts_tablet_manager.cc:1918] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:36.431711 17840 log.cc:1196] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/e22da709254f4adcbf55c9d5aabc0a5c
I20260811 06:56:36.432298 17841 ts_tablet_manager.cc:1918] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:36.433074 17840 ts_tablet_manager.cc:1939] T e22da709254f4adcbf55c9d5aabc0a5c P 04bed8119bb44f1580b28e08aa5132ac: Deleting consensus metadata
I20260811 06:56:36.433300 17841 log.cc:1196] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/e22da709254f4adcbf55c9d5aabc0a5c
I20260811 06:56:36.433964 17841 ts_tablet_manager.cc:1939] T e22da709254f4adcbf55c9d5aabc0a5c P 78862879db8d48389ead036510fb4094: Deleting consensus metadata
I20260811 06:56:36.437743 17842 log.cc:1196] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/e22da709254f4adcbf55c9d5aabc0a5c
I20260811 06:56:36.438405 17842 ts_tablet_manager.cc:1939] T e22da709254f4adcbf55c9d5aabc0a5c P e0cb6e18a37446a4ba804fb73c3077ac: Deleting consensus metadata
I20260811 06:56:36.452725 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.396966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33294 (local address 127.4.40.60:37825)
0811 06:56:36.397131 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.397136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.423874 (+ 26738us) server_negotiation.cc:407] Connection header received
0811 06:56:36.424262 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.424269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.424355 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.424430 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.426374 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.427507 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.429389 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.430057 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.430158 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.445728 (+ 15570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.445734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.445740 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.445781 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.449422 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.450248 (+ 826us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.450255 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.450259 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.450414 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.450856 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.450861 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.450864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.451241 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.451521 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.452061 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.452363 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:56:36.595808 15463 catalog_manager.cc:4841] TS e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375): tablet e22da709254f4adcbf55c9d5aabc0a5c (table test-table [id=5b9ca482fac84c0691e314829339bbb5]) successfully deleted
I20260811 06:56:36.597057 15467 catalog_manager.cc:4841] TS 78862879db8d48389ead036510fb4094 (127.4.40.3:42639): tablet e22da709254f4adcbf55c9d5aabc0a5c (table test-table [id=5b9ca482fac84c0691e314829339bbb5]) successfully deleted
I20260811 06:56:36.597358 15467 catalog_manager.cc:4841] TS 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877): tablet e22da709254f4adcbf55c9d5aabc0a5c (table test-table [id=5b9ca482fac84c0691e314829339bbb5]) successfully deleted
I20260811 06:56:36.597995 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 15434
I20260811 06:56:36.675595 17766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.645057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.4.40.61:40483)
0811 06:56:36.645305 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.645315 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.657810 (+ 12495us) server_negotiation.cc:407] Connection header received
0811 06:56:36.658023 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.658032 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.658163 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.658305 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.661347 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.662426 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.664263 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.664835 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.664970 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.668706 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.668712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.668717 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.668762 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.672368 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.673280 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.673285 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.673289 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.673460 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.673900 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.673905 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.673909 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.674214 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.674467 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.674828 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.675229 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:56:36.675678 17754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:36.644486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33296 (local address 127.4.40.60:37825)
0811 06:56:36.644731 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:36.644738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:36.657803 (+ 13065us) server_negotiation.cc:407] Connection header received
0811 06:56:36.658277 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:36.658283 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:36.658370 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:36.658442 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:36.661190 (+ 2748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.662382 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.664577 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:36.665236 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:36.665373 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:36.667991 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:36.668002 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:36.668011 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:36.668045 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:36.671758 (+ 3713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.672711 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.672721 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.672726 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.672872 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:36.673320 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:36.673326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:36.673330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:36.673743 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:36.674040 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:36.675066 (+ 1026us) server_negotiation.cc:299] Negotiation successful
0811 06:56:36.675303 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:56:37.355309 17862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.354292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:42359)
0811 06:56:37.354913 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.355087 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":382,"threads_started":1}
W20260811 06:56:37.355839 17487 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:45025 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.368563 17863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.366819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:51733)
0811 06:56:37.368207 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.368349 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1236,"thread_start_us":825,"threads_started":1}
W20260811 06:56:37.369136 17206 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:45025 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.370040 17865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.368283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:57075)
0811 06:56:37.369646 (+ 1363us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.369803 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1258,"threads_started":1}
W20260811 06:56:37.370649 17747 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:45025 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.884203 17868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d0c6a3f5057249c1b2cf914883ac53c9)
I20260811 06:56:37.884460 17868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:37.884732 17872 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d0c6a3f5057249c1b2cf914883ac53c9)
I20260811 06:56:37.885061 17872 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:37.885499 17868 leader_election.cc:288] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61:40483)
I20260811 06:56:37.893174 17872 leader_election.cc:288] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60:37825)
I20260811 06:56:37.898658 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.885579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41430)
0811 06:56:37.898090 (+ 12511us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.898291 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12319,"threads_started":1}
I20260811 06:56:37.900735 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.887579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41434)
0811 06:56:37.900140 (+ 12561us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.900321 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12402,"thread_start_us":12038,"threads_started":1}
W20260811 06:56:37.901039 16501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.909669 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.885864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:40483 (local address 127.0.0.1:45302)
0811 06:56:37.900146 (+ 14282us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.900171 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:37.900355 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:37.902877 (+ 2522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:37.902881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:37.902913 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:37.903212 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:37.903223 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.904929 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.904936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:37.907054 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:37.907067 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.908916 (+ 1849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.908923 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:37.909114 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:37.909145 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:56:37.909249 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:56:37.909386 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14128,"thread_start_us":12961,"threads_started":1}
I20260811 06:56:37.909658 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.893371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:37825 (local address 127.0.0.1:33304)
0811 06:56:37.902000 (+ 8629us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.902023 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:37.902196 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:37.902902 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:37.902906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:37.902927 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:37.903202 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:37.903213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.904910 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.904915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:37.907042 (+ 2127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:37.907057 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.908937 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.908943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:37.909124 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:37.909153 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:56:37.909262 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 06:56:37.909392 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8502,"threads_started":1}
I20260811 06:56:37.910102 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.909453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41456)
0811 06:56:37.909883 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.909968 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299}
W20260811 06:56:37.910284 16501 leader_election.cc:334] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
W20260811 06:56:37.910408 16850 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.910669 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.885929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45302 (local address 127.4.40.61:40483)
0811 06:56:37.902360 (+ 16431us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:37.902365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:37.902382 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:56:37.902440 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:37.902447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:37.902523 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:37.902632 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:37.903401 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.904782 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.907231 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.908588 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.908716 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:37.908811 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:37.910153 (+ 1342us) server_negotiation.cc:299] Negotiation successful
0811 06:56:37.910303 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16335,"thread_start_us":16162,"threads_started":1}
I20260811 06:56:37.911167 17881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.899129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.4.40.60:37825)
0811 06:56:37.901871 (+ 2742us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:37.901879 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:37.902459 (+ 580us) server_negotiation.cc:407] Connection header received
0811 06:56:37.902533 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:37.902537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:37.902623 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:37.902711 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:37.903401 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.904740 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.907217 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:37.908578 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:37.908716 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:37.908811 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:37.910623 (+ 1812us) server_negotiation.cc:299] Negotiation successful
0811 06:56:37.910757 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2587,"thread_start_us":2750,"threads_started":1}
I20260811 06:56:37.911324 16554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" is_pre_election: true
I20260811 06:56:37.911621 16554 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 in term 1.
I20260811 06:56:37.911934 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" is_pre_election: true
I20260811 06:56:37.912191 16846 leader_election.cc:302] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, f45dd7db5d6444a998c5afd3ede4fdc2; no voters:
I20260811 06:56:37.912338 16892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f45dd7db5d6444a998c5afd3ede4fdc2 in term 1.
I20260811 06:56:37.912684 17868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 06:56:37.912815 17868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d0c6a3f5057249c1b2cf914883ac53c9)
I20260811 06:56:37.912923 17868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:56:37.912851 16498 leader_election.cc:302] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, f45dd7db5d6444a998c5afd3ede4fdc2; no voters: d0c6a3f5057249c1b2cf914883ac53c9
I20260811 06:56:37.913272 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.912761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41468)
0811 06:56:37.912935 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.913034 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 06:56:37.913338 17872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 06:56:37.913452 17872 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d0c6a3f5057249c1b2cf914883ac53c9)
I20260811 06:56:37.913558 17872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 1 FOLLOWER]: Advancing to term 2
W20260811 06:56:37.913584 16850 leader_election.cc:334] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.916270 17868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:37.916604 17872 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:37.917058 17868 leader_election.cc:288] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 2 election: Requested vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), f45dd7db5d6444a998c5afd3ede4fdc2 (127.4.40.61:40483)
I20260811 06:56:37.917339 17872 leader_election.cc:288] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 2 election: Requested vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60:37825)
I20260811 06:56:37.917577 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.917115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41472)
0811 06:56:37.917331 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.917421 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 06:56:37.917783 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
I20260811 06:56:37.917865 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.917372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41476)
0811 06:56:37.917633 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.917734 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 06:56:37.918104 16892 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f45dd7db5d6444a998c5afd3ede4fdc2 in current term 2: Already voted for candidate 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 in this term.
I20260811 06:56:37.918169 16554 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2"
I20260811 06:56:37.918464 16554 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 in current term 2: Already voted for candidate f45dd7db5d6444a998c5afd3ede4fdc2 in this term.
I20260811 06:56:37.919119 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.918511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41486)
0811 06:56:37.918720 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.918911 (+ 191us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:56:37.919359 16501 leader_election.cc:334] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.919515 17875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:37.919105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:41488)
0811 06:56:37.919280 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:37.919365 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:56:37.919495 16501 leader_election.cc:302] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f45dd7db5d6444a998c5afd3ede4fdc2; no voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, d0c6a3f5057249c1b2cf914883ac53c9
W20260811 06:56:37.919727 16850 leader_election.cc:334] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:37.919803 17872 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 06:56:37.919929 16850 leader_election.cc:302] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1; no voters: d0c6a3f5057249c1b2cf914883ac53c9, f45dd7db5d6444a998c5afd3ede4fdc2
I20260811 06:56:37.920228 17868 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 06:56:38.357432 17886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:38.356570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:60131)
0811 06:56:38.357064 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:38.357189 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
I20260811 06:56:38.370672 17889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:38.369839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:55881)
0811 06:56:38.370319 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:38.370420 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":261,"threads_started":1}
I20260811 06:56:38.372005 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:38.371219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:42247)
0811 06:56:38.371761 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:38.371836 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":307,"threads_started":1}
I20260811 06:56:39.359253 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.358353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:35033)
0811 06:56:39.358880 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.359001 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":273,"threads_started":1}
I20260811 06:56:39.372884 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.371553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:48611)
0811 06:56:39.372597 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.372686 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":796,"threads_started":1}
I20260811 06:56:39.373412 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.372721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:51137)
0811 06:56:39.373187 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.373283 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":265,"threads_started":1}
I20260811 06:56:39.435906 17897 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:39.436167 17897 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:39.436880 17897 leader_election.cc:288] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60:37825)
I20260811 06:56:39.437546 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" is_pre_election: true
I20260811 06:56:39.437819 16892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f45dd7db5d6444a998c5afd3ede4fdc2 in term 2.
I20260811 06:56:39.438026 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.437095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51910)
0811 06:56:39.437637 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.437757 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":254,"threads_started":1}
I20260811 06:56:39.438308 16498 leader_election.cc:302] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, f45dd7db5d6444a998c5afd3ede4fdc2; no voters:
I20260811 06:56:39.438627 17897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 06:56:39.438784 17897 raft_consensus.cc:482] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:39.438890 17897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:56:39.439529 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.439129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51924)
0811 06:56:39.439303 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.439385 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 06:56:39.439760 16501 leader_election.cc:334] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:39.441499 17897 raft_consensus.cc:504] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:39.442112 17897 leader_election.cc:288] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 3 election: Requested vote from peers d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025), 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 (127.4.40.60:37825)
I20260811 06:56:39.442541 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.442145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51938)
0811 06:56:39.442351 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.442425 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 06:56:39.442665 16892 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1"
I20260811 06:56:39.442883 16892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:56:39.443512 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.443236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51944)
0811 06:56:39.443353 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.443411 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 06:56:39.443732 16501 leader_election.cc:334] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
I20260811 06:56:39.445263 16892 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f45dd7db5d6444a998c5afd3ede4fdc2 in term 3.
I20260811 06:56:39.445640 16498 leader_election.cc:302] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1, f45dd7db5d6444a998c5afd3ede4fdc2; no voters: d0c6a3f5057249c1b2cf914883ac53c9
I20260811 06:56:39.445951 17897 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:56:39.446339 17897 raft_consensus.cc:686] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [term 3 LEADER]: Becoming Leader. State: Replica: f45dd7db5d6444a998c5afd3ede4fdc2, State: Running, Role: LEADER
I20260811 06:56:39.446723 17897 consensus_queue.cc:237] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } }
I20260811 06:56:39.448752 17901 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f45dd7db5d6444a998c5afd3ede4fdc2. Latest consensus state: current_term: 3 leader_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:39.449100 17901 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:39.449728 17905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:39.453845 17905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:39.454725 17905 catalog_manager.cc:1259] Loaded cluster ID: 1d9658236d6049c2b98b8031f9e4b061
I20260811 06:56:39.454821 17905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:39.456133 17905 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:39.456956 17905 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Loaded TSK: 0
I20260811 06:56:39.457806 17905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:56:39.480902 16528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:56:39.488265 16528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:39.492642 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.492189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51952)
0811 06:56:39.492395 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.492491 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 06:56:39.492681 16892 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [term 3 FOLLOWER]: Refusing update from remote peer f45dd7db5d6444a998c5afd3ede4fdc2: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 06:56:39.493381 17901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 06:56:39.493777 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.493371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51954)
0811 06:56:39.493523 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.493637 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:56:39.494082 16501 consensus_peers.cc:487] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 -> Peer d0c6a3f5057249c1b2cf914883ac53c9 (127.4.40.62:45025): Couldn't send request to peer d0c6a3f5057249c1b2cf914883ac53c9. Status: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 06:56:39.496412 17907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f45dd7db5d6444a998c5afd3ede4fdc2. Latest consensus state: current_term: 3 leader_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:39.496687 17907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:56:39.497936 17901 sys_catalog.cc:455] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:39.498255 17901 sys_catalog.cc:458] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:39.499778 17908 mvcc.cc:205] Tried to move back new op lower bound from 7317223012314554368 to 7317223012137119744. Current Snapshot: MvccSnapshot[applied={T|T < 7317223012314554368}]
I20260811 06:56:39.499636 17907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "d0c6a3f5057249c1b2cf914883ac53c9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45025 } } peers { permanent_uuid: "f45dd7db5d6444a998c5afd3ede4fdc2" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 40483 } attrs { promote: false } } peers { permanent_uuid: "9b5bd6a8ad4841cbb7b84cd4c1ae2ab1" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 37825 } attrs { promote: false } } }
I20260811 06:56:39.500149 17907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b5bd6a8ad4841cbb7b84cd4c1ae2ab1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:56:39.520265 17923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.507249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44375 (local address 127.0.0.1:39244)
0811 06:56:39.512292 (+ 5043us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.512310 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.512429 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.513741 (+ 1312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.513745 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.513770 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.514201 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.514215 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.515807 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.515813 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.517837 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.517849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.519333 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.519339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.519522 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.519561 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.519661 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.519786 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4905,"mutex_wait_us":124,"thread_start_us":5484,"threads_started":1}
I20260811 06:56:39.521097 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.507522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47516 (local address 127.4.40.3:42639)
0811 06:56:39.508066 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.508076 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.513095 (+ 5019us) server_negotiation.cc:407] Connection header received
0811 06:56:39.513377 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.513387 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.513487 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.513593 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.514555 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.515688 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.518990 (+ 3302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.519942 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.520098 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.520265 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.520553 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.520707 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":322,"threads_started":1}
I20260811 06:56:39.521157 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.507565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39244 (local address 127.4.40.2:44375)
0811 06:56:39.513403 (+ 5838us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.513412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.513433 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:56:39.513511 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.513516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.513594 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.513700 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.514528 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.515693 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.518003 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.519140 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.520541 (+ 1401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.520657 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.520708 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.520855 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5678,"thread_start_us":5539,"threads_started":1}
I20260811 06:56:39.521858 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.506401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44518)
0811 06:56:39.506561 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.506583 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.506704 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.514063 (+ 7359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.514068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.514098 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.514594 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.514610 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.516060 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.516067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.518637 (+ 2570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.518650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.520773 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.520780 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.520977 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.521005 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.521106 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.521532 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":132}
I20260811 06:56:39.521867 17362 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6958dab07f4917844fe25ecfe82214 (DEFAULT_TABLE table=test-table [id=9565fdf4a8e8400d8ef767219dd6faee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:39.521885 17920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.507248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42639 (local address 127.0.0.1:47516)
0811 06:56:39.512990 (+ 5742us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.513008 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.513202 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.513848 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.513855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.513883 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.514344 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.514355 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.515901 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.515906 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.518854 (+ 2948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.518864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.520171 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.520177 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.520364 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.520401 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.521357 (+ 956us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.521476 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5619,"mutex_wait_us":718,"thread_start_us":4302,"threads_started":1}
I20260811 06:56:39.522334 17362 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6958dab07f4917844fe25ecfe82214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:39.523094 17623 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6958dab07f4917844fe25ecfe82214 (DEFAULT_TABLE table=test-table [id=9565fdf4a8e8400d8ef767219dd6faee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:39.523532 17623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6958dab07f4917844fe25ecfe82214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:39.524554 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.506548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44518 (local address 127.4.40.1:46877)
0811 06:56:39.513728 (+ 7180us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.513734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.513754 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:56:39.513824 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.513829 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.513921 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.514035 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.514756 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.515924 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.519119 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.520539 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.520698 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.520812 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.521153 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.521295 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7026,"thread_start_us":6894,"threads_started":1}
I20260811 06:56:39.529443 17088 tablet_service.cc:1467] Processing CreateTablet for tablet 1b6958dab07f4917844fe25ecfe82214 (DEFAULT_TABLE table=test-table [id=9565fdf4a8e8400d8ef767219dd6faee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:39.529855 17088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b6958dab07f4917844fe25ecfe82214. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:39.537030 17928 tablet_bootstrap.cc:492] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Bootstrap starting.
I20260811 06:56:39.538502 17928 tablet_bootstrap.cc:654] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:39.540632 17925 tablet_bootstrap.cc:492] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Bootstrap starting.
I20260811 06:56:39.541312 17929 tablet_bootstrap.cc:492] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac: Bootstrap starting.
I20260811 06:56:39.542114 17928 tablet_bootstrap.cc:492] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: No bootstrap required, opened a new log
I20260811 06:56:39.542315 17928 ts_tablet_manager.cc:1397] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:56:39.542837 17925 tablet_bootstrap.cc:654] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:39.543325 17928 raft_consensus.cc:348] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.543676 17928 raft_consensus.cc:374] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:39.543852 17928 raft_consensus.cc:729] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78862879db8d48389ead036510fb4094, State: Initialized, Role: FOLLOWER
I20260811 06:56:39.544180 17929 tablet_bootstrap.cc:654] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:39.544384 17928 consensus_queue.cc:260] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.550791 17925 tablet_bootstrap.cc:492] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: No bootstrap required, opened a new log
I20260811 06:56:39.550395 17929 tablet_bootstrap.cc:492] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac: No bootstrap required, opened a new log
I20260811 06:56:39.551148 17929 ts_tablet_manager.cc:1397] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260811 06:56:39.551180 17925 ts_tablet_manager.cc:1397] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20260811 06:56:39.551353 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.550883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:35781)
0811 06:56:39.551097 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.551202 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 06:56:39.552066 17925 raft_consensus.cc:348] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.551992 17929 raft_consensus.cc:348] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.552309 17925 raft_consensus.cc:374] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:39.552309 17929 raft_consensus.cc:374] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:39.552404 17929 raft_consensus.cc:729] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04bed8119bb44f1580b28e08aa5132ac, State: Initialized, Role: FOLLOWER
I20260811 06:56:39.552400 17925 raft_consensus.cc:729] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0cb6e18a37446a4ba804fb73c3077ac, State: Initialized, Role: FOLLOWER
I20260811 06:56:39.552654 17929 consensus_queue.cc:260] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.552675 17925 consensus_queue.cc:260] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.553795 17928 ts_tablet_manager.cc:1428] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Time spent starting tablet: real 0.011s user 0.002s sys 0.006s
I20260811 06:56:39.556787 17929 ts_tablet_manager.cc:1428] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260811 06:56:39.557025 17748 heartbeater.cc:503] Master 127.4.40.61:40483 was elected leader, sending a full tablet report...
I20260811 06:56:39.557130 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.556607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:54269)
0811 06:56:39.556841 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.556941 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 06:56:39.557700 17925 ts_tablet_manager.cc:1428] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 06:56:39.558322 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.557814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:54149)
0811 06:56:39.558033 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.558149 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 06:56:39.558765 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.558268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:39191)
0811 06:56:39.558489 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.558583 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 06:56:39.560935 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.560541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:34669)
0811 06:56:39.560717 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.560776 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 06:56:39.561442 17207 heartbeater.cc:503] Master 127.4.40.61:40483 was elected leader, sending a full tablet report...
I20260811 06:56:39.561630 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.561149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:48807)
0811 06:56:39.561365 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.561449 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 06:56:39.563226 17489 heartbeater.cc:503] Master 127.4.40.61:40483 was elected leader, sending a full tablet report...
I20260811 06:56:39.585307 17936 raft_consensus.cc:482] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:39.585542 17936 raft_consensus.cc:504] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.587180 17936 leader_election.cc:288] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375), 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:39.587706 17934 raft_consensus.cc:482] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:39.587981 17934 raft_consensus.cc:504] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.589986 17934 leader_election.cc:288] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877), 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:39.599071 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.587913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:39313 (local address 127.4.40.3:42639)
0811 06:56:39.588200 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.588209 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.589157 (+ 948us) server_negotiation.cc:407] Connection header received
0811 06:56:39.589383 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.589389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.589464 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.589578 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.590135 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.591176 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.595330 (+ 4154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.597901 (+ 2571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.597993 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.598080 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.598656 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.598793 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 06:56:39.600049 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.587626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42639 (local address 127.4.40.1:39313)
0811 06:56:39.589015 (+ 1389us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.589030 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.589219 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.589604 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.589610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.589640 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.589956 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.589966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.593205 (+ 3239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.593210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.595145 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.595159 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.598249 (+ 3090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.598255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.598444 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.598492 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.599504 (+ 1012us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.599672 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1280,"thread_start_us":810,"threads_started":1}
W20260811 06:56:39.600492 17751 tablet.cc:2369] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:39.600548 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.590355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:33379 (local address 127.4.40.1:46877)
0811 06:56:39.590525 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.590532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.591656 (+ 1124us) server_negotiation.cc:407] Connection header received
0811 06:56:39.591724 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.591728 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.591784 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.591872 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.592550 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.593542 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.595946 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.597835 (+ 1889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.597979 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.598080 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.600183 (+ 2103us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.600307 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":55}
I20260811 06:56:39.601083 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.590121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.4.40.2:33379)
0811 06:56:39.591408 (+ 1287us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.591424 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.591537 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.592014 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.592019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.592043 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.592349 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.592360 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.593711 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.593716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.595759 (+ 2043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.595775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.599620 (+ 3845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.599649 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.599854 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.599892 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.599989 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.600766 (+ 777us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1156}
I20260811 06:56:39.601328 17661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6958dab07f4917844fe25ecfe82214" candidate_uuid: "04bed8119bb44f1580b28e08aa5132ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78862879db8d48389ead036510fb4094" is_pre_election: true
I20260811 06:56:39.601930 17661 raft_consensus.cc:2454] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04bed8119bb44f1580b28e08aa5132ac in term 0.
I20260811 06:56:39.603087 17009 leader_election.cc:302] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bed8119bb44f1580b28e08aa5132ac, 78862879db8d48389ead036510fb4094; no voters:
I20260811 06:56:39.603726 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.587449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:52619 (local address 127.4.40.2:44375)
0811 06:56:39.587659 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.587666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.587967 (+ 301us) server_negotiation.cc:407] Connection header received
0811 06:56:39.588076 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.588081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.588201 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.588304 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.589317 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.590465 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.601271 (+ 10806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.602541 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.602641 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.602727 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.603321 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.603465 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:56:39.603914 17936 raft_consensus.cc:2781] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:39.604049 17936 raft_consensus.cc:482] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:39.604168 17936 raft_consensus.cc:3037] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:39.604743 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.587369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44375 (local address 127.4.40.1:52619)
0811 06:56:39.587653 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.587682 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.587833 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.588486 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.588491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.588538 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.589149 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.589164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.590645 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.590650 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.601095 (+ 10445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.601111 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.602923 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.602929 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.603132 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.603162 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.604191 (+ 1029us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.604419 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":8343}
I20260811 06:56:39.607530 17936 raft_consensus.cc:504] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.608342 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6958dab07f4917844fe25ecfe82214" candidate_uuid: "04bed8119bb44f1580b28e08aa5132ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" is_pre_election: true
I20260811 06:56:39.608623 17401 raft_consensus.cc:2454] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04bed8119bb44f1580b28e08aa5132ac in term 0.
I20260811 06:56:39.609896 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.590468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42639 (local address 127.4.40.2:45447)
0811 06:56:39.601298 (+ 10830us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.601316 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:39.601414 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:39.601853 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:39.601857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:39.601927 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:39.602068 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.602077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.603437 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.603442 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.606744 (+ 3302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:39.606760 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.608712 (+ 1952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.608718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:39.608970 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.609010 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 06:56:39.609103 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 06:56:39.609222 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10731,"thread_start_us":18711,"threads_started":1}
I20260811 06:56:39.610047 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.590766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:45447 (local address 127.4.40.3:42639)
0811 06:56:39.599458 (+ 8692us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.599464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.601517 (+ 2053us) server_negotiation.cc:407] Connection header received
0811 06:56:39.601583 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.601588 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.601653 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.601733 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:39.602236 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.603281 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.606937 (+ 3656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.608383 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.608526 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.608622 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.609582 (+ 960us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.609733 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8596,"thread_start_us":18366,"threads_started":1}
I20260811 06:56:39.610087 17124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6958dab07f4917844fe25ecfe82214" candidate_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04bed8119bb44f1580b28e08aa5132ac" is_pre_election: true
I20260811 06:56:39.610213 17661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6958dab07f4917844fe25ecfe82214" candidate_uuid: "04bed8119bb44f1580b28e08aa5132ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78862879db8d48389ead036510fb4094"
I20260811 06:56:39.610384 17124 raft_consensus.cc:2379] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e0cb6e18a37446a4ba804fb73c3077ac in current term 1: Already voted for candidate 04bed8119bb44f1580b28e08aa5132ac in this term.
I20260811 06:56:39.610441 17661 raft_consensus.cc:3037] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:39.610634 17401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6958dab07f4917844fe25ecfe82214" candidate_uuid: "04bed8119bb44f1580b28e08aa5132ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0cb6e18a37446a4ba804fb73c3077ac"
I20260811 06:56:39.610841 17401 raft_consensus.cc:3037] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:39.610889 17660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b6958dab07f4917844fe25ecfe82214" candidate_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78862879db8d48389ead036510fb4094" is_pre_election: true
I20260811 06:56:39.612214 17279 leader_election.cc:302] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e0cb6e18a37446a4ba804fb73c3077ac; no voters: 04bed8119bb44f1580b28e08aa5132ac, 78862879db8d48389ead036510fb4094
I20260811 06:56:39.612915 17936 leader_election.cc:288] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [CANDIDATE]: Term 1 election: Requested vote from peers e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375), 78862879db8d48389ead036510fb4094 (127.4.40.3:42639)
I20260811 06:56:39.613449 17401 raft_consensus.cc:2454] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04bed8119bb44f1580b28e08aa5132ac in term 1.
I20260811 06:56:39.613682 17934 raft_consensus.cc:2726] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 06:56:39.613677 17661 raft_consensus.cc:2454] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04bed8119bb44f1580b28e08aa5132ac in term 1.
I20260811 06:56:39.613933 17004 leader_election.cc:302] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04bed8119bb44f1580b28e08aa5132ac, e0cb6e18a37446a4ba804fb73c3077ac; no voters:
I20260811 06:56:39.614293 17936 raft_consensus.cc:2781] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:39.614749 17936 raft_consensus.cc:686] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 1 LEADER]: Becoming Leader. State: Replica: 04bed8119bb44f1580b28e08aa5132ac, State: Running, Role: LEADER
I20260811 06:56:39.615200 17936 consensus_queue.cc:237] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } }
I20260811 06:56:39.616084 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.615642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:54627)
0811 06:56:39.615849 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.615932 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 06:56:39.619247 16531 catalog_manager.cc:5495] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1). New cstate: current_term: 1 leader_uuid: "04bed8119bb44f1580b28e08aa5132ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04bed8119bb44f1580b28e08aa5132ac" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 46877 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 } health_report { overall_health: UNKNOWN } } }
I20260811 06:56:39.654452 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.649323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44526 (local address 127.4.40.1:46877)
0811 06:56:39.649487 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.649492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.649509 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:56:39.649658 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.649661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.649758 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.649877 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:39.650184 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.650955 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.652267 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.652750 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.652854 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.653304 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:39.653878 (+ 574us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:39.653985 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.654100 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.654207 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 06:56:39.658741 17401 raft_consensus.cc:1261] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Refusing update from remote peer 04bed8119bb44f1580b28e08aa5132ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:39.658941 17661 raft_consensus.cc:1261] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Refusing update from remote peer 04bed8119bb44f1580b28e08aa5132ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:39.659544 17948 consensus_queue.cc:1035] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0cb6e18a37446a4ba804fb73c3077ac" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:39.659890 17936 consensus_queue.cc:1035] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "78862879db8d48389ead036510fb4094" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:39.662499 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.661906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:46641)
0811 06:56:39.662226 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.662324 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260811 06:56:39.662706 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.661834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:40137)
0811 06:56:39.662391 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.662514 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463}
I20260811 06:56:39.667320 17896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.666813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.3:36283)
0811 06:56:39.667030 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.667119 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 06:56:39.668301 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.667865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:45389)
0811 06:56:39.668042 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.668161 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 06:56:39.675551 17894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.675041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.1:35735)
0811 06:56:39.675237 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.675365 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 06:56:39.678529 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.678051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.4.40.2:51303)
0811 06:56:39.678260 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.678344 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260811 06:56:39.681546 16531 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:39.681802 16531 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:39.691746 16531 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f45dd7db5d6444a998c5afd3ede4fdc2: Sending DeleteTablet for 3 replicas of tablet 1b6958dab07f4917844fe25ecfe82214
I20260811 06:56:39.693195 17623 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6958dab07f4917844fe25ecfe82214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:39 UTC) from {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:47516
I20260811 06:56:39.694180 17362 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6958dab07f4917844fe25ecfe82214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:39 UTC) from {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:39244
I20260811 06:56:39.694190 17088 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6958dab07f4917844fe25ecfe82214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:39 UTC) from {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:44518
I20260811 06:56:39.694648 17360 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6958dab07f4917844fe25ecfe82214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:39 UTC) from {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:39244
I20260811 06:56:39.695850 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 16845
I20260811 06:56:39.697134 17622 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b6958dab07f4917844fe25ecfe82214 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:39 UTC) from {username='oryx', principal='oryx/127.4.40.61@KRBTEST.COM'} at 127.0.0.1:47516
I20260811 06:56:39.717372 17918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.696744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44532 (local address 127.4.40.1:46877)
0811 06:56:39.696977 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.696985 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.699042 (+ 2057us) server_negotiation.cc:407] Connection header received
0811 06:56:39.699296 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.699302 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.699393 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.699485 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:39.701553 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.702713 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.704590 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.705278 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.705390 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.709065 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:39.709079 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:39.709099 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:39.709138 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:39.713685 (+ 4547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:39.714612 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:39.714622 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:39.714627 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:39.714787 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:39.715273 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:39.715278 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:39.715281 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:39.715814 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:39.716167 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.716789 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.717027 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:56:39.719164 17197 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:39.719789 17971 tablet_replica.cc:330] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: stopping tablet replica
I20260811 06:56:39.720036 17971 raft_consensus.cc:2229] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:39.720177 17973 tablet_replica.cc:330] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: stopping tablet replica
I20260811 06:56:39.720407 17973 raft_consensus.cc:2229] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:39.720597 17971 raft_consensus.cc:2258] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:39.720764 17973 raft_consensus.cc:2258] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:39.722270 17973 ts_tablet_manager.cc:1905] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:39.722311 17971 ts_tablet_manager.cc:1905] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:39.728520 17973 ts_tablet_manager.cc:1918] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:39.728672 17973 log.cc:1196] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/1b6958dab07f4917844fe25ecfe82214
I20260811 06:56:39.729096 17973 ts_tablet_manager.cc:1939] T 1b6958dab07f4917844fe25ecfe82214 P e0cb6e18a37446a4ba804fb73c3077ac: Deleting consensus metadata
I20260811 06:56:39.730132 17971 ts_tablet_manager.cc:1918] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:39.730226 16498 catalog_manager.cc:4841] TS e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375): tablet 1b6958dab07f4917844fe25ecfe82214 (table test-table [id=9565fdf4a8e8400d8ef767219dd6faee]) successfully deleted
I20260811 06:56:39.730400 17971 log.cc:1196] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/1b6958dab07f4917844fe25ecfe82214
W20260811 06:56:39.730829 16498 catalog_manager.cc:4808] TS e0cb6e18a37446a4ba804fb73c3077ac (127.4.40.2:44375): delete failed for tablet 1b6958dab07f4917844fe25ecfe82214 because the tablet was not found. No further retry: Not found: Tablet not found: 1b6958dab07f4917844fe25ecfe82214
I20260811 06:56:39.730912 17971 ts_tablet_manager.cc:1939] T 1b6958dab07f4917844fe25ecfe82214 P 78862879db8d48389ead036510fb4094: Deleting consensus metadata
I20260811 06:56:39.732081 16502 catalog_manager.cc:4841] TS 78862879db8d48389ead036510fb4094 (127.4.40.3:42639): tablet 1b6958dab07f4917844fe25ecfe82214 (table test-table [id=9565fdf4a8e8400d8ef767219dd6faee]) successfully deleted
W20260811 06:56:39.732455 16502 catalog_manager.cc:4808] TS 78862879db8d48389ead036510fb4094 (127.4.40.3:42639): delete failed for tablet 1b6958dab07f4917844fe25ecfe82214 because the tablet was not found. No further retry: Not found: Tablet not found: 1b6958dab07f4917844fe25ecfe82214
I20260811 06:56:39.824736 17972 tablet_replica.cc:330] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac: stopping tablet replica
I20260811 06:56:39.825309 17972 raft_consensus.cc:2229] T 1b6958dab07f4917844fe25ecfe82214 P 04bed8119bb44f1580b28e08aa5132ac [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:39.825618 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 16845
I20260811 06:56:39.853477 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 17233
I20260811 06:56:39.854413 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.853929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44548)
0811 06:56:39.854128 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.854226 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:56:39.854739 16502 catalog_manager.cc:4584] TS 04bed8119bb44f1580b28e08aa5132ac (127.4.40.1:46877): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b6958dab07f4917844fe25ecfe82214: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
I20260811 06:56:39.867794 17922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.854295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39250 (local address 127.4.40.2:44375)
0811 06:56:39.854513 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.854520 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.855086 (+ 566us) server_negotiation.cc:407] Connection header received
0811 06:56:39.855150 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.855154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.855269 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.855359 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:39.856839 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.857549 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.858778 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.859203 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.859332 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:39.862070 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:39.862092 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:39.862109 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:39.862143 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:39.865014 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:39.865675 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:39.865684 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:39.865687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:39.865827 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:39.866222 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:39.866227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:39.866231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:39.866604 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:39.866918 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:39.867203 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 06:56:39.867428 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 06:56:39.868520 17479 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:39.907660 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.907020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44564)
0811 06:56:39.907376 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.907468 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206}
I20260811 06:56:39.908782 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.908445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44580)
0811 06:56:39.908597 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.908672 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:56:39.963203 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 17233
I20260811 06:56:39.978338 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.977862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44596)
0811 06:56:39.978071 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.978161 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 06:56:39.979439 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.979125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44604)
0811 06:56:39.979263 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.979338 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:56:39.991159 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.990716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51970)
0811 06:56:39.990873 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.990978 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 06:56:39.992295 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.991933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45025 (local address 127.0.0.1:51982)
0811 06:56:39.992102 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:39.992201 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:45025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 06:56:39.993657 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 17504
I20260811 06:56:40.007516 17921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:39.994358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47526 (local address 127.4.40.3:42639)
0811 06:56:39.994532 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:39.994539 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:39.994637 (+ 98us) server_negotiation.cc:407] Connection header received
0811 06:56:39.994849 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:39.994857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:39.994931 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:39.995063 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:39.996548 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.997314 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.998547 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:39.998969 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:39.999117 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:40.001722 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:40.001738 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:40.001755 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:40.001792 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:40.004729 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:40.005400 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:40.005408 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:40.005410 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:40.005544 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:40.005927 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:40.005932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:40.005935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:40.006309 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:40.006648 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:40.006974 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 06:56:40.007240 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:56:40.008282 17738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:40.087631 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:40.087054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44610)
0811 06:56:40.087301 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:40.087411 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:56:40.088750 17899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:40.088363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:46877 (local address 127.0.0.1:44612)
0811 06:56:40.088522 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:40.088619 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:46877: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 06:56:40.129025 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 17504
I20260811 06:56:40.159312 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 16287
I20260811 06:56:40.160561 16591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:40.289588 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 16287
I20260811 06:56:40.325392 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 16464
I20260811 06:56:40.326550 16936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:40.446511 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 16464
2026-08-11T06:56:40Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11564 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:40 dist-test-slave-7vjv krb5kdc[17988](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:40 dist-test-slave-7vjv krb5kdc[17988](info): set up 2 sockets
Aug 11 06:56:40 dist-test-slave-7vjv krb5kdc[17988](info): commencing operation
krb5kdc: starting...
W20260811 06:56:41.807044 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.307s user 0.006s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:41 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:42Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab.
Entry for principal oryx/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab.
I20260811 06:56:42.070034 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:44549
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab
--principal=oryx/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:44549
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:42.405915 18009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:42.406221 18009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:42.406343 18009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:42.413339 18009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:42.413434 18009 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 06:56:42.413525 18009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:42.413655 18009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:42.413741 18009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:42.413853 18009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:42.423532 18009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:44549
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.62.keytab
--principal=oryx/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:44549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:42.424854 18009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:42.426144 18009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:42.438421 18018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:42.438339 18019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:42.438395 18021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:42.438953 18009 server_base.cc:1034] running on GCE node
I20260811 06:56:42.439787 18009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:42.441282 18009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:42.442528 18009 hybrid_clock.cc:648] HybridClock initialized: now 1786431402442462 us; error 42 us; skew 500 ppm
I20260811 06:56:42.442876 18009 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:42 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431402, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:42.449813 18009 init.cc:377] Logged in from keytab as oryx/127.4.40.62@KRBTEST.COM (short username oryx)
I20260811 06:56:42.451383 18009 webserver.cc:460] Webserver started at http://127.4.40.62:37399/ using document root <none> and password file <none>
I20260811 06:56:42.452184 18009 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:42.452320 18009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:42.452669 18009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:42.455778 18009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "b6f429ed9ee142dfa9d943bc817e6872"
format_stamp: "Formatted at 2026-08-11 06:56:42 on dist-test-slave-7vjv"
server_key: "8d889b3c70052a86ca98f6b1e139db58"
server_key_iv: "1c332e9c357be252567ac6fb1be0ae76"
server_key_version: "encryptionkey@0"
I20260811 06:56:42.456624 18009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "b6f429ed9ee142dfa9d943bc817e6872"
format_stamp: "Formatted at 2026-08-11 06:56:42 on dist-test-slave-7vjv"
server_key: "8d889b3c70052a86ca98f6b1e139db58"
server_key_iv: "1c332e9c357be252567ac6fb1be0ae76"
server_key_version: "encryptionkey@0"
I20260811 06:56:42.461736 18009 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:56:42.464982 18034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:42.466197 18009 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:56:42.466378 18009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "b6f429ed9ee142dfa9d943bc817e6872"
format_stamp: "Formatted at 2026-08-11 06:56:42 on dist-test-slave-7vjv"
server_key: "8d889b3c70052a86ca98f6b1e139db58"
server_key_iv: "1c332e9c357be252567ac6fb1be0ae76"
server_key_version: "encryptionkey@0"
I20260811 06:56:42.466599 18009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:42.495882 18009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:42.500052 18009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:42.500385 18009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:42.528247 18009 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:44549
I20260811 06:56:42.528262 18136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:44549 every 8 connection(s)
I20260811 06:56:42.529924 18009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:42.535159 18138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:42.536571 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 18009
I20260811 06:56:42.536839 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:42.537743 4256 external_mini_cluster.cc:1428] Setting key a7a2b1165a2f00ace0b2dc9bcb13f172
I20260811 06:56:42.551539 18138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Bootstrap starting.
I20260811 06:56:42.555804 18138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:42.557447 18138 log.cc:824] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:56:42 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431401, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:42.561236 18138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: No bootstrap required, opened a new log
I20260811 06:56:42.570003 18143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:42.540181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47098 (local address 127.4.40.62:44549)
0811 06:56:42.541112 (+ 931us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:42.541128 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:42.541200 (+ 72us) server_negotiation.cc:407] Connection header received
0811 06:56:42.542955 (+ 1755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:42.542982 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:42.543497 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:42.543918 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:42.545835 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:42.547934 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:42.549945 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:42.550931 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:42.551102 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:42.560600 (+ 9498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:42.560614 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:42.560637 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:42.560690 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:42.564861 (+ 4171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:42.565799 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:42.565808 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:42.565815 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:42.566015 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:42.566471 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:42.566477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:42.566480 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:42.567241 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:42.567545 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:42.568117 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 06:56:42.568570 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":406,"threads_started":1}
I20260811 06:56:42.571761 18138 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6f429ed9ee142dfa9d943bc817e6872" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44549 } }
I20260811 06:56:42.572144 18138 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:42.572270 18138 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6f429ed9ee142dfa9d943bc817e6872, State: Initialized, Role: FOLLOWER
I20260811 06:56:42.572799 18138 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6f429ed9ee142dfa9d943bc817e6872" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44549 } }
I20260811 06:56:42.573065 18138 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:42.573232 18138 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:42.573428 18138 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:42.575719 18138 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6f429ed9ee142dfa9d943bc817e6872" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44549 } }
I20260811 06:56:42.576309 18138 leader_election.cc:302] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6f429ed9ee142dfa9d943bc817e6872; no voters:
I20260811 06:56:42.576970 18138 leader_election.cc:288] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:42.577138 18145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:42.578065 18145 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [term 1 LEADER]: Becoming Leader. State: Replica: b6f429ed9ee142dfa9d943bc817e6872, State: Running, Role: LEADER
I20260811 06:56:42.578699 18145 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6f429ed9ee142dfa9d943bc817e6872" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44549 } }
I20260811 06:56:42.579393 18138 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:42.581948 18147 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6f429ed9ee142dfa9d943bc817e6872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6f429ed9ee142dfa9d943bc817e6872" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44549 } } }
I20260811 06:56:42.582585 18147 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:42.582093 18149 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6f429ed9ee142dfa9d943bc817e6872. Latest consensus state: current_term: 1 leader_uuid: "b6f429ed9ee142dfa9d943bc817e6872" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6f429ed9ee142dfa9d943bc817e6872" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 44549 } } }
I20260811 06:56:42.584292 18152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:42.584342 18149 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:42.589490 18152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:42.599723 18152 catalog_manager.cc:1347] Generated new cluster ID: 24e69f13058e4bc5ab32ecf982858e0b
I20260811 06:56:42.599857 18152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:42.612568 18152 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:42.613797 18152 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:42.623436 18152 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Generated new TSK 0
I20260811 06:56:42.624291 18152 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab.
Entry for principal oryx/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab.
I20260811 06:56:42.697376 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44549
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:43.070226 18184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:43.070552 18184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:43.070783 18184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:43.078300 18184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:43.078442 18184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:43.078579 18184 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:56:43.078714 18184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:43.088325 18184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:44549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:43.089574 18184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:43.090852 18184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:43.104362 18190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:43.104396 18194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:43.104460 18191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:43.105762 18184 server_base.cc:1034] running on GCE node
I20260811 06:56:43.106462 18184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:43.107331 18184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:43.108574 18184 hybrid_clock.cc:648] HybridClock initialized: now 1786431403108517 us; error 39 us; skew 500 ppm
I20260811 06:56:43.108868 18184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:43 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431403, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:43.112037 18184 init.cc:377] Logged in from keytab as oryx/127.4.40.1@KRBTEST.COM (short username oryx)
I20260811 06:56:43.113412 18184 webserver.cc:460] Webserver started at http://127.4.40.1:45679/ using document root <none> and password file <none>
I20260811 06:56:43.114111 18184 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:43.114245 18184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:43.114578 18184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:43.117519 18184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "3c4acf00ed764e05b6ccfd9184370781"
format_stamp: "Formatted at 2026-08-11 06:56:43 on dist-test-slave-7vjv"
server_key: "11bfdbfee32cf932f49ebd3c77b06c3f"
server_key_iv: "b4de94c05960d31011c499e196023bed"
server_key_version: "encryptionkey@0"
I20260811 06:56:43.118325 18184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "3c4acf00ed764e05b6ccfd9184370781"
format_stamp: "Formatted at 2026-08-11 06:56:43 on dist-test-slave-7vjv"
server_key: "11bfdbfee32cf932f49ebd3c77b06c3f"
server_key_iv: "b4de94c05960d31011c499e196023bed"
server_key_version: "encryptionkey@0"
I20260811 06:56:43.123287 18184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:56:43.126703 18207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:43.127995 18184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:56:43.128221 18184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "3c4acf00ed764e05b6ccfd9184370781"
format_stamp: "Formatted at 2026-08-11 06:56:43 on dist-test-slave-7vjv"
server_key: "11bfdbfee32cf932f49ebd3c77b06c3f"
server_key_iv: "b4de94c05960d31011c499e196023bed"
server_key_version: "encryptionkey@0"
I20260811 06:56:43.128456 18184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:43.146746 18184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:43.151099 18184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:43.151389 18184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:43.154145 18184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:43.154260 18184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:43.154480 18184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:43.154565 18184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:56:43 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431403, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:43.179811 18224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.155148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.4.40.1:40895)
0811 06:56:43.156069 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:43.156170 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:43.158356 (+ 2186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:43.159231 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:43.159242 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:43.160071 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:43.161024 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.161055 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.162521 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.162524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.164862 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.164885 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.166066 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.166081 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.166425 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.167240 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:43.167274 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:43.170789 (+ 3515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:43.175137 (+ 4348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.175148 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.175164 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.175955 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.176475 (+ 520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.176483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.176487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.176816 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.177605 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:43.177610 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:43.177926 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 06:56:43.178208 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 06:56:43.178623 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":407,"threads_started":1}
I20260811 06:56:43.181051 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.154524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:40895 (local address 127.4.40.62:44549)
0811 06:56:43.155069 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:43.155075 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:43.156308 (+ 1233us) server_negotiation.cc:407] Connection header received
0811 06:56:43.158681 (+ 2373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:43.158688 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:43.158806 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:43.158950 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:43.161277 (+ 2327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.162349 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.165041 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.165757 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.165947 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.171117 (+ 5170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:43.171127 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:43.171135 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:43.171190 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:43.174943 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.176132 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.176140 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.176145 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.176346 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.176993 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.177002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.177007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.177457 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:43.177657 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:43.180574 (+ 2917us) server_negotiation.cc:299] Negotiation successful
0811 06:56:43.180800 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":270,"threads_started":1}
I20260811 06:56:43.206382 18184 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:37143
I20260811 06:56:43.206491 18414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:37143 every 8 connection(s)
I20260811 06:56:43.207724 18184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:43.209649 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 18184
I20260811 06:56:43.209872 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:43.210328 18418 heartbeater.cc:347] Connected to a master server at 127.4.40.62:44549
I20260811 06:56:43.210583 18418 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:43.210729 4256 external_mini_cluster.cc:1428] Setting key 3b95f1d4c906d318deb497165d9a4615
I20260811 06:56:43.211386 18418 heartbeater.cc:511] Master 127.4.40.62:44549 requested a full tablet report, sending...
I20260811 06:56:43.216650 18065 ts_manager.cc:194] Registered new tserver with Master: 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143)
I20260811 06:56:43.217036 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.198544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:49893 (local address 127.4.40.62:44549)
0811 06:56:43.198740 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:43.198747 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:43.199628 (+ 881us) server_negotiation.cc:407] Connection header received
0811 06:56:43.199956 (+ 328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:43.199963 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:43.200060 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:43.200250 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:43.201750 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.202783 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.204878 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.205564 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.205685 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.208125 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:43.208132 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:43.208136 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:43.208174 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:43.213789 (+ 5615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.214648 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.214654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.214657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.214811 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.215246 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.215251 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.215253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.215675 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:43.215850 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:43.216434 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 06:56:43.216708 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 06:56:43.217262 18385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.198650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.4.40.1:49893)
0811 06:56:43.199481 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:43.199509 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:43.199698 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:43.200279 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:43.200284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:43.201033 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:43.201589 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.201609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.202922 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.202928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.204730 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.204744 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.205971 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.205980 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.206194 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.207009 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:43.207033 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:43.207885 (+ 852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:43.213950 (+ 6065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.213974 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.213978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.214527 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.214943 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.214950 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.214963 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.215133 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.215823 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:43.215831 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:43.216089 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 06:56:43.216315 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 06:56:43.216699 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":527,"threads_started":1}
I20260811 06:56:43.219429 18065 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:40895
WARNING: no policy specified for oryx/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab.
Entry for principal oryx/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab.
I20260811 06:56:43.272838 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44549
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:43.610726 18425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:43.611027 18425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:43.611200 18425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:43.618198 18425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:43.618340 18425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:43.618477 18425 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:56:43.618611 18425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:43.628082 18425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:44549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:43.629335 18425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:43.630658 18425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:43.643272 18434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:43.643242 18438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:43.643318 18436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:43.644750 18425 server_base.cc:1034] running on GCE node
I20260811 06:56:43.645352 18425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:43.646217 18425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:43.647431 18425 hybrid_clock.cc:648] HybridClock initialized: now 1786431403647376 us; error 36 us; skew 500 ppm
I20260811 06:56:43.647728 18425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:43 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431403, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:43.650943 18425 init.cc:377] Logged in from keytab as oryx/127.4.40.2@KRBTEST.COM (short username oryx)
I20260811 06:56:43.652441 18425 webserver.cc:460] Webserver started at http://127.4.40.2:37903/ using document root <none> and password file <none>
I20260811 06:56:43.653134 18425 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:43.653268 18425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:43.653594 18425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:43.656443 18425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "461ad0ff78f34b4887fe619ccad25d0e"
format_stamp: "Formatted at 2026-08-11 06:56:43 on dist-test-slave-7vjv"
server_key: "b836048bfa132763ee0732e6ea9d64d5"
server_key_iv: "283ab7db15ac6b4202972bfebd4e0148"
server_key_version: "encryptionkey@0"
I20260811 06:56:43.657238 18425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "461ad0ff78f34b4887fe619ccad25d0e"
format_stamp: "Formatted at 2026-08-11 06:56:43 on dist-test-slave-7vjv"
server_key: "b836048bfa132763ee0732e6ea9d64d5"
server_key_iv: "283ab7db15ac6b4202972bfebd4e0148"
server_key_version: "encryptionkey@0"
I20260811 06:56:43.661978 18425 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 06:56:43.665163 18458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:43.666421 18425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:43.666596 18425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "461ad0ff78f34b4887fe619ccad25d0e"
format_stamp: "Formatted at 2026-08-11 06:56:43 on dist-test-slave-7vjv"
server_key: "b836048bfa132763ee0732e6ea9d64d5"
server_key_iv: "283ab7db15ac6b4202972bfebd4e0148"
server_key_version: "encryptionkey@0"
I20260811 06:56:43.666807 18425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:43.681334 18425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:43.685762 18425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:43.686038 18425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:43.688725 18425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:43.688848 18425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:43.689050 18425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:43.689152 18425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:56:43 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431403, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:43.712797 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.689428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:47895 (local address 127.4.40.62:44549)
0811 06:56:43.689778 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:43.689788 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:43.690931 (+ 1143us) server_negotiation.cc:407] Connection header received
0811 06:56:43.693062 (+ 2131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:43.693067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:43.693161 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:43.693287 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:43.695416 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.696483 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.699374 (+ 2891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.699808 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.699946 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.704982 (+ 5036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:43.704991 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:43.704997 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:43.705046 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:43.709037 (+ 3991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.709950 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.709959 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.709965 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.710145 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.710598 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.710605 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.710610 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.711105 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:43.711317 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:43.711968 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 06:56:43.712281 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260811 06:56:43.713745 18472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.689878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.4.40.2:47895)
0811 06:56:43.690765 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:43.690807 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:43.692800 (+ 1993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:43.693566 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:43.693573 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:43.694221 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:43.695209 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.695243 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.696680 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.696686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.699223 (+ 2537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.699235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.700066 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.700074 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.700314 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.701220 (+ 906us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:43.701258 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:43.704690 (+ 3432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:43.709209 (+ 4519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.709225 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.709242 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.709819 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.710274 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.710278 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.710280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.710483 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.711269 (+ 786us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:43.711274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:43.711576 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 06:56:43.711801 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 06:56:43.712331 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":431,"threads_started":1}
I20260811 06:56:43.749807 18599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.726391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.4.40.2:50687)
0811 06:56:43.732478 (+ 6087us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:43.732504 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:43.732695 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:43.733231 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:43.733235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:43.733991 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:43.734533 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.734549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.735910 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.735914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.737179 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:43.737195 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.737961 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.737970 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:43.738248 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.739080 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:43.739100 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:43.740405 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:43.744281 (+ 3876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.744288 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.744294 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.747133 (+ 2839us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.747830 (+ 697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:43.747834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:43.747836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:43.747977 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:43.748858 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:43.748862 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:43.749026 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 06:56:43.749132 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:56:43.749362 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5901,"thread_start_us":5759,"threads_started":1}
I20260811 06:56:43.750087 18221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:43.726394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:50687 (local address 127.4.40.62:44549)
0811 06:56:43.726608 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:43.726616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:43.732638 (+ 6022us) server_negotiation.cc:407] Connection header received
0811 06:56:43.732978 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:43.732985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:43.733081 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:43.733184 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:43.734695 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.735776 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.737328 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:43.737759 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:43.737901 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:43.740619 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:43.740625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:43.740629 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:43.740668 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:43.744070 (+ 3402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.747454 (+ 3384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.747463 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.747468 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.747677 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:43.748142 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:43.748150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:43.748156 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:43.748729 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:43.748915 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:43.749298 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 06:56:43.749606 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:56:43.754747 18425 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:35785
I20260811 06:56:43.754796 18687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:35785 every 8 connection(s)
I20260811 06:56:43.755931 18425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:43.757855 18688 heartbeater.cc:347] Connected to a master server at 127.4.40.62:44549
I20260811 06:56:43.758101 18688 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:43.758539 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 18425
I20260811 06:56:43.758718 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:43.758833 18688 heartbeater.cc:511] Master 127.4.40.62:44549 requested a full tablet report, sending...
I20260811 06:56:43.759256 4256 external_mini_cluster.cc:1428] Setting key 921c2ea1d0390d49c42d18ccc0b74eff
I20260811 06:56:43.760293 18065 ts_manager.cc:194] Registered new tserver with Master: 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785)
I20260811 06:56:43.761325 18065 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:47895
WARNING: no policy specified for oryx/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab.
Entry for principal oryx/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab.
I20260811 06:56:43.821103 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:44549
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:44.156591 18694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:44.156840 18694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:44.157015 18694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:44.163934 18694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:44.164032 18694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:44.164129 18694 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 06:56:44.164259 18694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:44.173897 18694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.40.62:44549
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:44.175092 18694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:44.176420 18694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:44.189730 18701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:44.189694 18705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:44.189774 18702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:44.193683 18694 server_base.cc:1034] running on GCE node
I20260811 06:56:44.194255 18694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:44.195180 18694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:44.196421 18694 hybrid_clock.cc:648] HybridClock initialized: now 1786431404196361 us; error 36 us; skew 500 ppm
I20260811 06:56:44.196708 18694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431404, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:44.199939 18694 init.cc:377] Logged in from keytab as oryx/127.4.40.3@KRBTEST.COM (short username oryx)
I20260811 06:56:44.201478 18694 webserver.cc:460] Webserver started at http://127.4.40.3:34803/ using document root <none> and password file <none>
I20260811 06:56:44.202131 18694 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:44.202215 18694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:44.202486 18694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:44.205363 18694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "d22d6f250579411c820fde809c1a275c"
format_stamp: "Formatted at 2026-08-11 06:56:44 on dist-test-slave-7vjv"
server_key: "bad256cf9b291de79b44487c129928bd"
server_key_iv: "53ba67b1a1329d635e0e6a46968aca29"
server_key_version: "encryptionkey@0"
I20260811 06:56:44.206197 18694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "d22d6f250579411c820fde809c1a275c"
format_stamp: "Formatted at 2026-08-11 06:56:44 on dist-test-slave-7vjv"
server_key: "bad256cf9b291de79b44487c129928bd"
server_key_iv: "53ba67b1a1329d635e0e6a46968aca29"
server_key_version: "encryptionkey@0"
I20260811 06:56:44.211305 18694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 06:56:44.214993 18713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:44.216929 18694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 06:56:44.217144 18694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "d22d6f250579411c820fde809c1a275c"
format_stamp: "Formatted at 2026-08-11 06:56:44 on dist-test-slave-7vjv"
server_key: "bad256cf9b291de79b44487c129928bd"
server_key_iv: "53ba67b1a1329d635e0e6a46968aca29"
server_key_version: "encryptionkey@0"
I20260811 06:56:44.217453 18694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:44.222751 18418 heartbeater.cc:503] Master 127.4.40.62:44549 was elected leader, sending a full tablet report...
I20260811 06:56:44.272357 18694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:44.276949 18694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:44.277257 18694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:44.280028 18694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:44.280174 18694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 06:56:44.280334 18694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:44.280411 18694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431404, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:44.304183 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.280859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38401 (local address 127.4.40.62:44549)
0811 06:56:44.283754 (+ 2895us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.283782 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.283810 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:56:44.286457 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.286464 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.286578 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.286716 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:44.289490 (+ 2774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.290518 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.292351 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.292814 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.292947 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.297393 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:44.297402 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:44.297409 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:44.297449 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:44.301135 (+ 3686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.301835 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.301841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.301845 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.301977 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.302408 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.302414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.302422 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.302689 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:44.302888 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.303560 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.303763 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2782,"thread_start_us":2630,"threads_started":1}
I20260811 06:56:44.305212 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.281688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.4.40.3:38401)
0811 06:56:44.282798 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.282868 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.286077 (+ 3209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.287072 (+ 995us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.287088 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.288162 (+ 1074us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:44.289255 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.289284 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.290753 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.290761 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.292190 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.292202 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.293084 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.293092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.293303 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.294037 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:44.294061 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:44.297065 (+ 3004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:44.301292 (+ 4227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:44.301300 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:44.301314 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:44.301705 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:44.302092 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:44.302097 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:44.302101 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:44.302286 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:44.302849 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:44.302856 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:44.303191 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.303455 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.303931 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":481,"threads_started":1}
I20260811 06:56:44.330644 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.315734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38603 (local address 127.4.40.62:44549)
0811 06:56:44.315891 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.315895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.317130 (+ 1235us) server_negotiation.cc:407] Connection header received
0811 06:56:44.317344 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.317349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.317417 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.317524 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:44.318548 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.319234 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.320797 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.321687 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.321865 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.323714 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:44.323723 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:44.323731 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:44.323777 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:44.327736 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.328531 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.328539 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.328543 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.328716 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.329223 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.329227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.329229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.329496 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:44.329647 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.330228 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.330392 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 06:56:44.331265 18822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.315912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.4.40.3:38603)
0811 06:56:44.317016 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.317030 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.317190 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.317545 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.317550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.318110 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:44.318399 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.318408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.319388 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.319393 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.320600 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.320632 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.321900 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.321913 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.322117 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.322795 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:44.322811 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:44.323487 (+ 676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:44.327879 (+ 4392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:44.327886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:44.327889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:44.328390 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:44.328861 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:44.328873 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:44.328880 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:44.329084 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:44.329677 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:44.329684 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:44.329954 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.330142 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.330578 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":887,"thread_start_us":691,"threads_started":1}
I20260811 06:56:44.346807 18694 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:41503
I20260811 06:56:44.346843 18919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:41503 every 8 connection(s)
I20260811 06:56:44.347990 18694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:44.348630 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 18694
I20260811 06:56:44.348821 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:44.349466 4256 external_mini_cluster.cc:1428] Setting key 90f87ce5b10337cdb16e625638b30297
I20260811 06:56:44.350869 18920 heartbeater.cc:347] Connected to a master server at 127.4.40.62:44549
I20260811 06:56:44.351112 18920 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:44.351751 18920 heartbeater.cc:511] Master 127.4.40.62:44549 requested a full tablet report, sending...
I20260811 06:56:44.353152 18064 ts_manager.cc:194] Registered new tserver with Master: d22d6f250579411c820fde809c1a275c (127.4.40.3:41503)
I20260811 06:56:44.353956 18064 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:38401
I20260811 06:56:44.364866 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431404, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:56:44.426534 18729 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:44.415876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47106 (local address 127.4.40.62:44549)
0811 06:56:44.416065 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.416069 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.416369 (+ 300us) server_negotiation.cc:407] Connection header received
0811 06:56:44.416551 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.416559 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.416648 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.416790 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:44.418172 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.418884 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.420165 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.420556 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.420652 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.426162 (+ 5510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47106: BlockingRecv error: recv got EOF from 127.0.0.1:47106 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431404, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:44.442198 18729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.429491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.4.40.62:44549)
0811 06:56:44.429691 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.429696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.429904 (+ 208us) server_negotiation.cc:407] Connection header received
0811 06:56:44.430030 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.430033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.430088 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.430165 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:44.431174 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.431866 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.433166 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.433686 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.433781 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.437281 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:44.437285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:44.437288 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:44.437317 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:44.439970 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.440578 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.440582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.440584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.440707 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.441042 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.441045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.441047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.441300 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:44.441495 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.441713 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.441872 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:56:44.446138 18064 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:56:44.453276 18064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:44.476791 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.466304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.4.40.2:35785)
0811 06:56:44.467228 (+ 924us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.467237 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.468615 (+ 1378us) server_negotiation.cc:407] Connection header received
0811 06:56:44.468944 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.468951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.469159 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.469400 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:44.470399 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.471879 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.474109 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.475297 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.475432 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.475649 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.476331 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.476493 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":1599,"threads_started":1}
I20260811 06:56:44.477213 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.465669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:41503 (local address 127.0.0.1:47114)
0811 06:56:44.466480 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.466528 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.466709 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.468604 (+ 1895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.468619 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.468672 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:44.469163 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.469178 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.471745 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.471749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.474153 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.474172 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.475979 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.475985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.476144 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.476176 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.476263 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.476461 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":73,"spinlock_wait_cycles":4480,"thread_start_us":2553,"threads_started":1}
I20260811 06:56:44.477447 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.465927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47114 (local address 127.4.40.3:41503)
0811 06:56:44.467880 (+ 1953us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.467890 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.467928 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:56:44.468027 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.468034 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.468323 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.469370 (+ 1047us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:44.469763 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.471616 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.474332 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.475755 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.476630 (+ 875us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.476862 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.476938 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.477117 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1775,"thread_start_us":1465,"threads_started":1}
I20260811 06:56:44.477492 18952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.465719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35785 (local address 127.0.0.1:59604)
0811 06:56:44.468499 (+ 2780us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.468516 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.468715 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.469674 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.469681 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.469715 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:44.470234 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.470246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.472082 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.472112 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.473883 (+ 1771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.473895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.475796 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.475802 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.476048 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.476128 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.476257 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.476499 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2652,"mutex_wait_us":88,"thread_start_us":726,"threads_started":1}
I20260811 06:56:44.480048 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.467723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60824 (local address 127.4.40.1:37143)
0811 06:56:44.469572 (+ 1849us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.469582 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.470232 (+ 650us) server_negotiation.cc:407] Connection header received
0811 06:56:44.470558 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.470566 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.470729 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.470899 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:44.471534 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.473434 (+ 1900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.475962 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.477477 (+ 1515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.479270 (+ 1793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.479520 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.479600 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.479759 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1701,"thread_start_us":1556,"threads_started":1}
I20260811 06:56:44.480399 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.467167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37143 (local address 127.0.0.1:60824)
0811 06:56:44.470070 (+ 2903us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.470092 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.470291 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.470893 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.470897 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.470920 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:44.471335 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.471348 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.473588 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.473593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.475805 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.475822 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.478483 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.478490 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.478706 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.478737 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.478806 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.478953 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2746,"thread_start_us":2601,"threads_started":1}
I20260811 06:56:44.481082 18546 tablet_service.cc:1467] Processing CreateTablet for tablet ee3c7e16b3594d4090e08d35da5f76f7 (DEFAULT_TABLE table=test-table [id=100b25717d01482d89a748a040589fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:44.481079 18801 tablet_service.cc:1467] Processing CreateTablet for tablet ee3c7e16b3594d4090e08d35da5f76f7 (DEFAULT_TABLE table=test-table [id=100b25717d01482d89a748a040589fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:44.482398 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3c7e16b3594d4090e08d35da5f76f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:44.482517 18801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3c7e16b3594d4090e08d35da5f76f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:44.483839 18309 tablet_service.cc:1467] Processing CreateTablet for tablet ee3c7e16b3594d4090e08d35da5f76f7 (DEFAULT_TABLE table=test-table [id=100b25717d01482d89a748a040589fab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:44.485154 18309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3c7e16b3594d4090e08d35da5f76f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:44.493746 18967 tablet_bootstrap.cc:492] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Bootstrap starting.
I20260811 06:56:44.497068 18969 tablet_bootstrap.cc:492] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Bootstrap starting.
I20260811 06:56:44.497622 18967 tablet_bootstrap.cc:654] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:44.498998 18967 log.cc:824] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:44.499462 18970 tablet_bootstrap.cc:492] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Bootstrap starting.
I20260811 06:56:44.500705 18969 tablet_bootstrap.cc:654] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:44.501978 18969 log.cc:824] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:44.502812 18967 tablet_bootstrap.cc:492] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: No bootstrap required, opened a new log
I20260811 06:56:44.502831 18970 tablet_bootstrap.cc:654] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:44.503088 18967 ts_tablet_manager.cc:1397] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 06:56:44.503834 18970 log.cc:824] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:44.504920 18969 tablet_bootstrap.cc:492] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: No bootstrap required, opened a new log
I20260811 06:56:44.505213 18969 ts_tablet_manager.cc:1397] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.006s
I20260811 06:56:44.506654 18970 tablet_bootstrap.cc:492] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: No bootstrap required, opened a new log
I20260811 06:56:44.506920 18970 ts_tablet_manager.cc:1397] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 06:56:44.513938 18967 raft_consensus.cc:348] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.514453 18967 raft_consensus.cc:374] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:44.514561 18967 raft_consensus.cc:729] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Initialized, Role: FOLLOWER
I20260811 06:56:44.514653 18970 raft_consensus.cc:348] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.515146 18970 raft_consensus.cc:374] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:44.515283 18970 raft_consensus.cc:729] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4acf00ed764e05b6ccfd9184370781, State: Initialized, Role: FOLLOWER
I20260811 06:56:44.515261 18967 consensus_queue.cc:260] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.515954 18970 consensus_queue.cc:260] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.515960 18969 raft_consensus.cc:348] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.516393 18969 raft_consensus.cc:374] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:44.516487 18969 raft_consensus.cc:729] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22d6f250579411c820fde809c1a275c, State: Initialized, Role: FOLLOWER
I20260811 06:56:44.517104 18969 consensus_queue.cc:260] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.517390 18967 ts_tablet_manager.cc:1428] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260811 06:56:44.517946 18688 heartbeater.cc:503] Master 127.4.40.62:44549 was elected leader, sending a full tablet report...
I20260811 06:56:44.520340 18920 heartbeater.cc:503] Master 127.4.40.62:44549 was elected leader, sending a full tablet report...
I20260811 06:56:44.521106 18969 ts_tablet_manager.cc:1428] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Time spent starting tablet: real 0.016s user 0.016s sys 0.001s
I20260811 06:56:44.522688 18970 ts_tablet_manager.cc:1428] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Time spent starting tablet: real 0.016s user 0.013s sys 0.004s
W20260811 06:56:44.599849 18922 tablet.cc:2369] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:44.651284 18982 raft_consensus.cc:482] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:44.651592 18982 raft_consensus.cc:504] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.653293 18982 leader_election.cc:288] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785), 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143)
I20260811 06:56:44.659919 18956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.653814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:46809 (local address 127.4.40.2:35785)
0811 06:56:44.654045 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.654052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.654114 (+ 62us) server_negotiation.cc:407] Connection header received
0811 06:56:44.654201 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.654208 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.654300 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.654401 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:44.654942 (+ 541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.656012 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.657984 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.659080 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.659180 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.659257 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.659637 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.659728 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:56:44.660028 18730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.653574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35785 (local address 127.4.40.3:46809)
0811 06:56:44.653769 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.653785 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.653930 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.654454 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.654473 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.654508 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:44.654724 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.654746 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.656173 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.656179 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.657827 (+ 1648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.657839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.659231 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.659236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.659438 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.659489 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.659595 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.659726 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":18}
I20260811 06:56:44.660912 18966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.654124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38507 (local address 127.4.40.1:37143)
0811 06:56:44.654348 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.654354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.654741 (+ 387us) server_negotiation.cc:407] Connection header received
0811 06:56:44.654924 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.654936 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.655014 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.655084 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:44.655727 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.656766 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.658857 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.659929 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.660079 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.660202 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.660593 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.660722 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 06:56:44.660983 18983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.653947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37143 (local address 127.4.40.3:38507)
0811 06:56:44.654619 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.654636 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.654763 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.655127 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.655132 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.655157 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:44.655540 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.655552 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.656923 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.656928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.658691 (+ 1763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.658708 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.660300 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.660309 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.660435 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.660457 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.660536 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.660616 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":9,"thread_start_us":321,"threads_started":1}
I20260811 06:56:44.661327 18597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3c7e16b3594d4090e08d35da5f76f7" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461ad0ff78f34b4887fe619ccad25d0e" is_pre_election: true
I20260811 06:56:44.661913 18597 raft_consensus.cc:2454] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 0.
I20260811 06:56:44.661952 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3c7e16b3594d4090e08d35da5f76f7" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781" is_pre_election: true
I20260811 06:56:44.662508 18345 raft_consensus.cc:2454] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 0.
I20260811 06:56:44.662580 18718 leader_election.cc:302] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461ad0ff78f34b4887fe619ccad25d0e, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:44.663081 18982 raft_consensus.cc:2781] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:44.663231 18982 raft_consensus.cc:482] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:44.663355 18982 raft_consensus.cc:3037] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:44.665757 18982 raft_consensus.cc:504] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.666322 18982 leader_election.cc:288] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 election: Requested vote from peers 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785), 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143)
I20260811 06:56:44.666749 18597 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3c7e16b3594d4090e08d35da5f76f7" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461ad0ff78f34b4887fe619ccad25d0e"
I20260811 06:56:44.666828 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3c7e16b3594d4090e08d35da5f76f7" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781"
I20260811 06:56:44.666950 18597 raft_consensus.cc:3037] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:44.667063 18345 raft_consensus.cc:3037] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:44.669343 18597 raft_consensus.cc:2454] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 1.
I20260811 06:56:44.669358 18345 raft_consensus.cc:2454] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 1.
I20260811 06:56:44.669744 18718 leader_election.cc:302] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4acf00ed764e05b6ccfd9184370781, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:44.670126 18982 raft_consensus.cc:2781] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:44.670629 18982 raft_consensus.cc:686] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 1 LEADER]: Becoming Leader. State: Replica: d22d6f250579411c820fde809c1a275c, State: Running, Role: LEADER
I20260811 06:56:44.671182 18982 consensus_queue.cc:237] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } }
I20260811 06:56:44.675527 18064 catalog_manager.cc:5495] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c reported cstate change: term changed from 0 to 1, leader changed from <none> to d22d6f250579411c820fde809c1a275c (127.4.40.3). New cstate: current_term: 1 leader_uuid: "d22d6f250579411c820fde809c1a275c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } health_report { overall_health: UNKNOWN } } }
W20260811 06:56:44.710687 18420 tablet.cc:2369] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:44.715210 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.709843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47124 (local address 127.4.40.3:41503)
0811 06:56:44.710042 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.710046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.710065 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:56:44.710223 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.710228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.710317 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.710511 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:44.710833 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.711812 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.713111 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.713532 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.713628 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.714036 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:44.714649 (+ 613us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:44.714756 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.714839 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.714901 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:56:44.722937 18345 raft_consensus.cc:1261] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Refusing update from remote peer d22d6f250579411c820fde809c1a275c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:44.723023 18597 raft_consensus.cc:1261] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Refusing update from remote peer d22d6f250579411c820fde809c1a275c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:44.723918 18984 consensus_queue.cc:1035] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:44.724314 18982 consensus_queue.cc:1035] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [LEADER]: Connected to new peer: Peer: permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:44.739835 18993 mvcc.cc:205] Tried to move back new op lower bound from 7317223033735888896 to 7317223033536876544. Current Snapshot: MvccSnapshot[applied={T|T < 7317223033735888896}]
I20260811 06:56:44.742107 18997 mvcc.cc:205] Tried to move back new op lower bound from 7317223033735888896 to 7317223033536876544. Current Snapshot: MvccSnapshot[applied={T|T < 7317223033735888896}]
I20260811 06:56:44.753216 18065 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:44.753492 18065 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:44.757596 18065 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Sending DeleteTablet for 3 replicas of tablet ee3c7e16b3594d4090e08d35da5f76f7
I20260811 06:56:44.758752 18801 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3c7e16b3594d4090e08d35da5f76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:44 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:47114
I20260811 06:56:44.761420 18309 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3c7e16b3594d4090e08d35da5f76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:44 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:60824
I20260811 06:56:44.762343 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3c7e16b3594d4090e08d35da5f76f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:44 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59604
I20260811 06:56:44.768647 19010 tablet_replica.cc:330] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: stopping tablet replica
I20260811 06:56:44.768450 18065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47110:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:56:44.769083 19010 raft_consensus.cc:2229] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260811 06:56:44.769172 18065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:44.769697 19010 raft_consensus.cc:2258] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:44.770162 19007 tablet_replica.cc:330] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: stopping tablet replica
I20260811 06:56:44.770608 19007 raft_consensus.cc:2229] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:44.777019 19010 ts_tablet_manager.cc:1905] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:44.777333 19007 raft_consensus.cc:2258] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:44.777391 18305 tablet_service.cc:1467] Processing CreateTablet for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a (DEFAULT_TABLE table=test-table [id=759de4dc4a15483db9ec2b0b328d079d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:44.777981 18305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:44.780639 19009 tablet_replica.cc:330] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: stopping tablet replica
I20260811 06:56:44.780571 18801 tablet_service.cc:1467] Processing CreateTablet for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a (DEFAULT_TABLE table=test-table [id=759de4dc4a15483db9ec2b0b328d079d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:44.781132 19009 raft_consensus.cc:2229] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:44.781193 18801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:44.781639 19009 raft_consensus.cc:2258] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:44.782675 18546 tablet_service.cc:1467] Processing CreateTablet for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a (DEFAULT_TABLE table=test-table [id=759de4dc4a15483db9ec2b0b328d079d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:44.783142 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:44.786953 18970 tablet_bootstrap.cc:492] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Bootstrap starting.
I20260811 06:56:44.788684 18970 tablet_bootstrap.cc:654] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:44.788865 18969 tablet_bootstrap.cc:492] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Bootstrap starting.
I20260811 06:56:44.790463 18969 tablet_bootstrap.cc:654] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:44.791613 18967 tablet_bootstrap.cc:492] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Bootstrap starting.
I20260811 06:56:44.791875 19007 ts_tablet_manager.cc:1905] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:44.793288 18967 tablet_bootstrap.cc:654] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:44.794867 19009 ts_tablet_manager.cc:1905] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:44.795279 19010 ts_tablet_manager.cc:1918] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:44.795491 19010 log.cc:1196] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/ee3c7e16b3594d4090e08d35da5f76f7
I20260811 06:56:44.796214 19010 ts_tablet_manager.cc:1939] T ee3c7e16b3594d4090e08d35da5f76f7 P 461ad0ff78f34b4887fe619ccad25d0e: Deleting consensus metadata
I20260811 06:56:44.803160 18970 tablet_bootstrap.cc:492] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: No bootstrap required, opened a new log
I20260811 06:56:44.803336 18970 ts_tablet_manager.cc:1397] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260811 06:56:44.803635 18041 catalog_manager.cc:4841] TS 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785): tablet ee3c7e16b3594d4090e08d35da5f76f7 (table test-table [id=100b25717d01482d89a748a040589fab]) successfully deleted
I20260811 06:56:44.804323 18970 raft_consensus.cc:348] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.804569 18970 raft_consensus.cc:374] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:44.804673 18970 raft_consensus.cc:729] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4acf00ed764e05b6ccfd9184370781, State: Initialized, Role: FOLLOWER
I20260811 06:56:44.805013 18970 consensus_queue.cc:260] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.808138 18970 ts_tablet_manager.cc:1428] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:56:44.810349 19009 ts_tablet_manager.cc:1918] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:44.810541 19009 log.cc:1196] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/ee3c7e16b3594d4090e08d35da5f76f7
I20260811 06:56:44.810686 19007 ts_tablet_manager.cc:1918] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:44.810907 19007 log.cc:1196] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/ee3c7e16b3594d4090e08d35da5f76f7
I20260811 06:56:44.811146 19009 ts_tablet_manager.cc:1939] T ee3c7e16b3594d4090e08d35da5f76f7 P 3c4acf00ed764e05b6ccfd9184370781: Deleting consensus metadata
I20260811 06:56:44.811501 19007 ts_tablet_manager.cc:1939] T ee3c7e16b3594d4090e08d35da5f76f7 P d22d6f250579411c820fde809c1a275c: Deleting consensus metadata
I20260811 06:56:44.813185 18041 catalog_manager.cc:4841] TS 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143): tablet ee3c7e16b3594d4090e08d35da5f76f7 (table test-table [id=100b25717d01482d89a748a040589fab]) successfully deleted
I20260811 06:56:44.813283 18036 catalog_manager.cc:4841] TS d22d6f250579411c820fde809c1a275c (127.4.40.3:41503): tablet ee3c7e16b3594d4090e08d35da5f76f7 (table test-table [id=100b25717d01482d89a748a040589fab]) successfully deleted
I20260811 06:56:44.813997 18967 tablet_bootstrap.cc:492] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: No bootstrap required, opened a new log
I20260811 06:56:44.814203 18967 ts_tablet_manager.cc:1397] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Time spent bootstrapping tablet: real 0.023s user 0.008s sys 0.003s
I20260811 06:56:44.814908 18969 tablet_bootstrap.cc:492] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: No bootstrap required, opened a new log
I20260811 06:56:44.815073 18969 ts_tablet_manager.cc:1397] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.008s
I20260811 06:56:44.815110 18967 raft_consensus.cc:348] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.815393 18967 raft_consensus.cc:374] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:44.815497 18967 raft_consensus.cc:729] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Initialized, Role: FOLLOWER
I20260811 06:56:44.815794 18967 consensus_queue.cc:260] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.816021 18969 raft_consensus.cc:348] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.816272 18969 raft_consensus.cc:374] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:44.816380 18969 raft_consensus.cc:729] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22d6f250579411c820fde809c1a275c, State: Initialized, Role: FOLLOWER
I20260811 06:56:44.816560 18967 ts_tablet_manager.cc:1428] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 06:56:44.816723 18969 consensus_queue.cc:260] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.817399 18969 ts_tablet_manager.cc:1428] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:56:44.912624 18984 raft_consensus.cc:482] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:44.912925 18984 raft_consensus.cc:504] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.913693 18984 leader_election.cc:288] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785)
I20260811 06:56:44.914294 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bcd30c904dd4a18a68e8b3d02da7f0a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781" is_pre_election: true
I20260811 06:56:44.914572 18345 raft_consensus.cc:2454] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 0.
I20260811 06:56:44.914381 18595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bcd30c904dd4a18a68e8b3d02da7f0a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461ad0ff78f34b4887fe619ccad25d0e" is_pre_election: true
I20260811 06:56:44.914793 18595 raft_consensus.cc:2454] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 0.
I20260811 06:56:44.915042 18718 leader_election.cc:302] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4acf00ed764e05b6ccfd9184370781, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:44.915345 18984 raft_consensus.cc:2781] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:44.915477 18984 raft_consensus.cc:482] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:44.915638 18984 raft_consensus.cc:3037] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:44.918077 18984 raft_consensus.cc:504] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.918617 18984 leader_election.cc:288] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 election: Requested vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785)
I20260811 06:56:44.919166 18595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bcd30c904dd4a18a68e8b3d02da7f0a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461ad0ff78f34b4887fe619ccad25d0e"
I20260811 06:56:44.919209 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5bcd30c904dd4a18a68e8b3d02da7f0a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781"
I20260811 06:56:44.919373 18595 raft_consensus.cc:3037] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:44.919432 18345 raft_consensus.cc:3037] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:44.921736 18595 raft_consensus.cc:2454] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 1.
I20260811 06:56:44.922212 18718 leader_election.cc:302] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461ad0ff78f34b4887fe619ccad25d0e, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:44.922533 18984 raft_consensus.cc:2781] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:44.922602 18345 raft_consensus.cc:2454] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 1.
I20260811 06:56:44.922734 18984 raft_consensus.cc:686] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 1 LEADER]: Becoming Leader. State: Replica: d22d6f250579411c820fde809c1a275c, State: Running, Role: LEADER
I20260811 06:56:44.922995 18984 consensus_queue.cc:237] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:44.925896 18065 catalog_manager.cc:5495] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c reported cstate change: term changed from 0 to 1, leader changed from <none> to d22d6f250579411c820fde809c1a275c (127.4.40.3). New cstate: current_term: 1 leader_uuid: "d22d6f250579411c820fde809c1a275c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } health_report { overall_health: HEALTHY } } }
Aug 11 06:56:44 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431402, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.62@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:44.989319 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.970961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:44549 (local address 127.0.0.1:47120)
0811 06:56:44.971822 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:44.971853 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:44.972007 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:44.972435 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:44.972442 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:44.973410 (+ 968us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:44.973796 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.973809 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.974888 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.974893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.976628 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:44.976644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.977412 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.977420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:44.977601 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.981011 (+ 3410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:44.981029 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:44.983354 (+ 2325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:44.986645 (+ 3291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:44.986659 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:44.986667 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:44.987098 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:44.987484 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:44.987490 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:44.987493 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:44.987626 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:44.988171 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:44.988178 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:44.988352 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 06:56:44.988521 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 06:56:44.988824 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"thread_start_us":534,"threads_started":1}
I20260811 06:56:44.989317 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:44.970904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.4.40.62:44549)
0811 06:56:44.971699 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:44.971705 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:44.971944 (+ 239us) server_negotiation.cc:407] Connection header received
0811 06:56:44.972139 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:44.972167 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:44.972261 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:44.972352 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:44.973969 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.974734 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.976778 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:44.977211 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:44.977307 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:44.983538 (+ 6231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:44.983543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:44.983546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:44.983573 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:44.986507 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.987224 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.987231 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.987239 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.987366 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:44.987732 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:44.987738 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:44.987741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:44.988013 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:44.988190 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:44.988607 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 06:56:44.988951 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":586,"thread_start_us":316,"threads_started":1}
I20260811 06:56:44.994685 18065 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:47120:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 06:56:44.995687 18065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:45.003129 18305 tablet_service.cc:1467] Processing CreateTablet for tablet 81f126d34a554ccdbfaec1aab2bec23f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ff3cf28361c4aac96b966366432d7fc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:56:45.003309 18546 tablet_service.cc:1467] Processing CreateTablet for tablet 81f126d34a554ccdbfaec1aab2bec23f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ff3cf28361c4aac96b966366432d7fc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:56:45.003763 18305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f126d34a554ccdbfaec1aab2bec23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.003844 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f126d34a554ccdbfaec1aab2bec23f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.004654 18801 tablet_service.cc:1467] Processing CreateTablet for tablet 81f126d34a554ccdbfaec1aab2bec23f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ff3cf28361c4aac96b966366432d7fc]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 06:56:45.005239 18801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81f126d34a554ccdbfaec1aab2bec23f. 1 dirs total, 0 dirs full, 0 dirs failed
W20260811 06:56:45.009711 18689 tablet.cc:2369] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:56:45.011372 18967 tablet_bootstrap.cc:492] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: Bootstrap starting.
I20260811 06:56:45.012861 18969 tablet_bootstrap.cc:492] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: Bootstrap starting.
I20260811 06:56:45.012928 18967 tablet_bootstrap.cc:654] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.014966 18969 tablet_bootstrap.cc:654] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.017230 18970 tablet_bootstrap.cc:492] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: Bootstrap starting.
I20260811 06:56:45.019017 18970 tablet_bootstrap.cc:654] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.023288 18970 tablet_bootstrap.cc:492] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: No bootstrap required, opened a new log
I20260811 06:56:45.023516 18970 ts_tablet_manager.cc:1397] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260811 06:56:45.023901 18969 tablet_bootstrap.cc:492] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: No bootstrap required, opened a new log
I20260811 06:56:45.024127 18969 ts_tablet_manager.cc:1397] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 06:56:45.024290 18970 raft_consensus.cc:348] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.024559 18970 raft_consensus.cc:374] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.024643 18970 raft_consensus.cc:729] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4acf00ed764e05b6ccfd9184370781, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.024842 18969 raft_consensus.cc:348] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.024888 18970 consensus_queue.cc:260] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.025097 18969 raft_consensus.cc:374] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.025192 18969 raft_consensus.cc:729] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22d6f250579411c820fde809c1a275c, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.025544 18970 ts_tablet_manager.cc:1428] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 06:56:45.025501 18969 consensus_queue.cc:260] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.025410 18981 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.025835 18981 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.025995 18984 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.026243 18984 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.026667 18967 tablet_bootstrap.cc:492] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: No bootstrap required, opened a new log
I20260811 06:56:45.026830 18967 ts_tablet_manager.cc:1397] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.010s
I20260811 06:56:45.027894 18969 ts_tablet_manager.cc:1428] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:56:45.028663 18967 raft_consensus.cc:348] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.028929 18967 raft_consensus.cc:374] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.029002 18967 raft_consensus.cc:729] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.029320 18967 consensus_queue.cc:260] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.029784 18978 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.030122 18978 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.031304 18967 ts_tablet_manager.cc:1428] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 06:56:45.162988 18978 raft_consensus.cc:482] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:45.163275 18978 raft_consensus.cc:504] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.164961 18978 leader_election.cc:288] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), d22d6f250579411c820fde809c1a275c (127.4.40.3:41503)
I20260811 06:56:45.173418 19042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.165461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:48905 (local address 127.4.40.1:37143)
0811 06:56:45.166406 (+ 945us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.166414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.166437 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:56:45.166532 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.166536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.166616 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.166747 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:45.167441 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.168714 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.171597 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.172411 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.172571 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.172673 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.172934 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.173084 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":809,"thread_start_us":684,"threads_started":1}
I20260811 06:56:45.173719 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.165144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37143 (local address 127.4.40.2:48905)
0811 06:56:45.165674 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:45.165696 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:45.165841 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:45.166793 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:45.166801 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:45.166867 (+ 66us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:45.167222 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.167234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.168862 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.168869 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.171371 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.171387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.172584 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.172590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.172767 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.172792 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:56:45.172897 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:56:45.173027 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":62,"spinlock_wait_cycles":20992,"thread_start_us":471,"threads_started":1}
I20260811 06:56:45.173995 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.165747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:34677 (local address 127.4.40.3:41503)
0811 06:56:45.165929 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.165935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.166746 (+ 811us) server_negotiation.cc:407] Connection header received
0811 06:56:45.167060 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.167064 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.167143 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.167241 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:56:45.167783 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.168586 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.171549 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.172822 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.172940 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.173030 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.173571 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.173699 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 06:56:45.174477 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f126d34a554ccdbfaec1aab2bec23f" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781" is_pre_election: true
I20260811 06:56:45.174762 19041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.165277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:41503 (local address 127.4.40.2:34677)
0811 06:56:45.166635 (+ 1358us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:45.166656 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:45.166885 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:45.167263 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:45.167267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:45.167292 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:56:45.167602 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.167611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.168728 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.168736 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.171371 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.171387 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.173190 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.173196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.173369 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.173401 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:56:45.174241 (+ 840us) client_negotiation.cc:240] Negotiation successful
0811 06:56:45.174379 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1245,"mutex_wait_us":127,"spinlock_wait_cycles":11648,"thread_start_us":1095,"threads_started":1}
I20260811 06:56:45.174782 18345 raft_consensus.cc:2454] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 0.
I20260811 06:56:45.175319 18465 leader_election.cc:302] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4acf00ed764e05b6ccfd9184370781, 461ad0ff78f34b4887fe619ccad25d0e; no voters:
I20260811 06:56:45.175592 18847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f126d34a554ccdbfaec1aab2bec23f" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22d6f250579411c820fde809c1a275c" is_pre_election: true
I20260811 06:56:45.175770 18978 raft_consensus.cc:2781] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:45.175904 18978 raft_consensus.cc:482] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:45.175993 18978 raft_consensus.cc:3037] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.176054 18847 raft_consensus.cc:2454] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 0.
I20260811 06:56:45.179248 18978 raft_consensus.cc:504] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.179975 18978 leader_election.cc:288] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 election: Requested vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), d22d6f250579411c820fde809c1a275c (127.4.40.3:41503)
I20260811 06:56:45.180528 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f126d34a554ccdbfaec1aab2bec23f" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781"
I20260811 06:56:45.180567 18847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81f126d34a554ccdbfaec1aab2bec23f" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22d6f250579411c820fde809c1a275c"
I20260811 06:56:45.180799 18345 raft_consensus.cc:3037] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.180814 18847 raft_consensus.cc:3037] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.184024 18847 raft_consensus.cc:2454] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 1.
I20260811 06:56:45.184015 18345 raft_consensus.cc:2454] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 1.
I20260811 06:56:45.184490 18459 leader_election.cc:302] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461ad0ff78f34b4887fe619ccad25d0e, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:45.184852 18978 raft_consensus.cc:2781] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:45.185346 18978 raft_consensus.cc:686] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [term 1 LEADER]: Becoming Leader. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Running, Role: LEADER
I20260811 06:56:45.186024 18978 consensus_queue.cc:237] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.188272 19045 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: TxnStatusTablet state changed. Reason: New leader 461ad0ff78f34b4887fe619ccad25d0e. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.188764 19045 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:56:45.189618 19048 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 06:56:45.189863 19048 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 06:56:45.190207 18061 catalog_manager.cc:5495] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2). New cstate: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } health_report { overall_health: UNKNOWN } } }
I20260811 06:56:45.243221 19028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.235732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35785 (local address 127.0.0.1:59618)
0811 06:56:45.236009 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:45.236030 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:45.236207 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:45.236925 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:45.236930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:45.236943 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:56:45.237197 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.237207 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.238876 (+ 1669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.238883 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.240579 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.240595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.241359 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.241366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.241561 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.241638 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.242632 (+ 994us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.242666 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:56:45.242762 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 06:56:45.242910 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 06:56:45.243260 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.233698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.4.40.2:35785)
0811 06:56:45.236438 (+ 2740us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.236444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.236472 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:56:45.236576 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.236582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.236705 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.236881 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:45.237403 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.238710 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.240727 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.241175 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.241277 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.241865 (+ 588us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.242481 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.242681 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.242815 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.242899 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2582,"thread_start_us":2656,"threads_started":1}
I20260811 06:56:45.247890 18345 raft_consensus.cc:1261] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Refusing update from remote peer 461ad0ff78f34b4887fe619ccad25d0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:45.248128 18847 raft_consensus.cc:1261] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Refusing update from remote peer 461ad0ff78f34b4887fe619ccad25d0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:45.248998 19045 consensus_queue.cc:1035] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:45.249300 18978 consensus_queue.cc:1035] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:45.253909 18981 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: TxnStatusTablet state changed. Reason: New leader 461ad0ff78f34b4887fe619ccad25d0e. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.254304 18981 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.257910 19045 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.258188 19045 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:56:45.258309 19055 mvcc.cc:205] Tried to move back new op lower bound from 7317223035888492544 to 7317223035646087168. Current Snapshot: MvccSnapshot[applied={T|T < 7317223035888492544}]
I20260811 06:56:45.267802 18981 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.268059 18981 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 3c4acf00ed764e05b6ccfd9184370781: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.270080 18984 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: TxnStatusTablet state changed. Reason: New leader 461ad0ff78f34b4887fe619ccad25d0e. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.270358 18984 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.271224 19045 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.271555 19045 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P 461ad0ff78f34b4887fe619ccad25d0e: This TxnStatusTablet replica's current role is: LEADER
I20260811 06:56:45.280867 18984 tablet_replica.cc:439] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } } }
I20260811 06:56:45.281236 18984 tablet_replica.cc:442] T 81f126d34a554ccdbfaec1aab2bec23f P d22d6f250579411c820fde809c1a275c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 06:56:45.300771 19071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.294301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35785 (local address 127.4.40.3:42911)
0811 06:56:45.295006 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:45.295021 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:45.295136 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:45.295504 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:45.295507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:45.295516 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:56:45.295753 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.295759 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.297344 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.297348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.298434 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.298446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.299652 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.299657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.299772 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.299821 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.300301 (+ 480us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.300318 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 06:56:45.300382 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 06:56:45.300478 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":407,"threads_started":1}
I20260811 06:56:45.300905 19052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.294456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:42911 (local address 127.4.40.2:35785)
0811 06:56:45.294615 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.294619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.295120 (+ 501us) server_negotiation.cc:407] Connection header received
0811 06:56:45.295259 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.295266 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.295381 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.295504 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:45.295918 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.297204 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.298579 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.299460 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.299650 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.300014 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.300194 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.300312 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.300419 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.300510 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:56:45.310128 18345 raft_consensus.cc:1261] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Refusing update from remote peer d22d6f250579411c820fde809c1a275c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:45.310288 18595 raft_consensus.cc:1261] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Refusing update from remote peer d22d6f250579411c820fde809c1a275c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:45.310755 18984 consensus_queue.cc:1035] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:45.311283 18982 consensus_queue.cc:1035] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [LEADER]: Connected to new peer: Peer: permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:45.350668 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.344155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:58915 (local address 127.4.40.3:41503)
0811 06:56:45.344344 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.344361 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.345154 (+ 793us) server_negotiation.cc:407] Connection header received
0811 06:56:45.345371 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.345377 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.345451 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.345547 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:45.346101 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.347041 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.348833 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.349337 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.349484 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.349914 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.350008 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.350140 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.350286 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.350360 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:56:45.350806 19083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.344256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:41503 (local address 127.4.40.2:58915)
0811 06:56:45.345057 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:45.345078 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:45.345239 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:45.345568 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:45.345572 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:45.345584 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:56:45.345951 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.345959 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.347218 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.347224 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.348688 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:45.348702 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.349523 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.349529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:45.349675 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.349725 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.350144 (+ 419us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.350157 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 06:56:45.350262 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 06:56:45.350394 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":374,"threads_started":1}
I20260811 06:56:45.399864 18061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:45.400151 18061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47110:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:45.403009 18061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Sending DeleteTablet for 3 replicas of tablet 5bcd30c904dd4a18a68e8b3d02da7f0a
I20260811 06:56:45.403797 18305 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:45 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:60824
I20260811 06:56:45.403822 18801 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:45 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:47114
I20260811 06:56:45.404547 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet 5bcd30c904dd4a18a68e8b3d02da7f0a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:45 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59604
I20260811 06:56:45.404594 19085 tablet_replica.cc:330] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: stopping tablet replica
I20260811 06:56:45.404850 19085 raft_consensus.cc:2229] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:45.405155 19085 raft_consensus.cc:2258] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:45.406562 19085 ts_tablet_manager.cc:1905] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:45.416539 19087 tablet_replica.cc:330] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: stopping tablet replica
I20260811 06:56:45.416837 19087 raft_consensus.cc:2229] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:45.417352 19087 raft_consensus.cc:2258] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:45.418308 19085 ts_tablet_manager.cc:1918] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:56:45.418497 19085 log.cc:1196] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/5bcd30c904dd4a18a68e8b3d02da7f0a
I20260811 06:56:45.418875 19087 ts_tablet_manager.cc:1905] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:45.419144 19085 ts_tablet_manager.cc:1939] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 3c4acf00ed764e05b6ccfd9184370781: Deleting consensus metadata
I20260811 06:56:45.420580 18041 catalog_manager.cc:4841] TS 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143): tablet 5bcd30c904dd4a18a68e8b3d02da7f0a (table test-table [id=759de4dc4a15483db9ec2b0b328d079d]) successfully deleted
I20260811 06:56:45.425961 19089 tablet_replica.cc:330] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: stopping tablet replica
I20260811 06:56:45.426272 19089 raft_consensus.cc:2229] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:45.426537 19087 ts_tablet_manager.cc:1918] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:56:45.426695 19089 raft_consensus.cc:2258] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:45.426751 19087 log.cc:1196] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/5bcd30c904dd4a18a68e8b3d02da7f0a
I20260811 06:56:45.427415 19087 ts_tablet_manager.cc:1939] T 5bcd30c904dd4a18a68e8b3d02da7f0a P d22d6f250579411c820fde809c1a275c: Deleting consensus metadata
I20260811 06:56:45.428020 19089 ts_tablet_manager.cc:1905] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:45.428874 18036 catalog_manager.cc:4841] TS d22d6f250579411c820fde809c1a275c (127.4.40.3:41503): tablet 5bcd30c904dd4a18a68e8b3d02da7f0a (table test-table [id=759de4dc4a15483db9ec2b0b328d079d]) successfully deleted
I20260811 06:56:45.434830 19089 ts_tablet_manager.cc:1918] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:56:45.435005 19089 log.cc:1196] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/5bcd30c904dd4a18a68e8b3d02da7f0a
I20260811 06:56:45.435525 19089 ts_tablet_manager.cc:1939] T 5bcd30c904dd4a18a68e8b3d02da7f0a P 461ad0ff78f34b4887fe619ccad25d0e: Deleting consensus metadata
I20260811 06:56:45.436743 18041 catalog_manager.cc:4841] TS 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785): tablet 5bcd30c904dd4a18a68e8b3d02da7f0a (table test-table [id=759de4dc4a15483db9ec2b0b328d079d]) successfully deleted
Aug 11 06:56:45 dist-test-slave-7vjv krb5kdc[17988](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 06:56:45 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:45 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:56:45.492935 19026 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:45.480741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47122 (local address 127.4.40.62:44549)
0811 06:56:45.480986 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.480993 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.481592 (+ 599us) server_negotiation.cc:407] Connection header received
0811 06:56:45.481782 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.481788 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.481879 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.482003 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:45.484193 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.485357 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.486687 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.487372 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.487508 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.492655 (+ 5147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47122: BlockingRecv error: recv got EOF from 127.0.0.1:47122 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 06:56:45 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM
I20260811 06:56:45.512010 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.495712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47138 (local address 127.4.40.62:44549)
0811 06:56:45.495861 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.495865 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.496374 (+ 509us) server_negotiation.cc:407] Connection header received
0811 06:56:45.496516 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.496519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.496580 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.496659 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:45.498129 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.498894 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.500717 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.501134 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.501226 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.505647 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:45.505670 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:45.505673 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:45.505705 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:45.509257 (+ 3552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:45.509897 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:45.509903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:45.509907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:45.510113 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:45.510472 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:45.510477 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:45.510481 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:45.510854 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:45.511136 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.511402 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.511643 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 06:56:45.515699 18061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:56:45.516423 18061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:45.523322 18305 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb5847d395b46d5b104ebbce449d29a (DEFAULT_TABLE table=test-table [id=24a5b6ed7b344b509053936be1701d09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:45.523689 18801 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb5847d395b46d5b104ebbce449d29a (DEFAULT_TABLE table=test-table [id=24a5b6ed7b344b509053936be1701d09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:45.523844 18305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb5847d395b46d5b104ebbce449d29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.524135 18801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb5847d395b46d5b104ebbce449d29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.530694 18970 tablet_bootstrap.cc:492] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Bootstrap starting.
I20260811 06:56:45.531340 18546 tablet_service.cc:1467] Processing CreateTablet for tablet 4fb5847d395b46d5b104ebbce449d29a (DEFAULT_TABLE table=test-table [id=24a5b6ed7b344b509053936be1701d09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:45.531800 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4fb5847d395b46d5b104ebbce449d29a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.532402 18970 tablet_bootstrap.cc:654] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.534996 18970 tablet_bootstrap.cc:492] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: No bootstrap required, opened a new log
I20260811 06:56:45.535143 18970 ts_tablet_manager.cc:1397] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 06:56:45.535904 18970 raft_consensus.cc:348] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.536194 18970 raft_consensus.cc:374] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.536284 18970 raft_consensus.cc:729] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4acf00ed764e05b6ccfd9184370781, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.536532 18970 consensus_queue.cc:260] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.539372 18970 ts_tablet_manager.cc:1428] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:56:45.541941 19111 tablet_bootstrap.cc:492] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Bootstrap starting.
I20260811 06:56:45.543823 19111 tablet_bootstrap.cc:654] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.544940 19109 tablet_bootstrap.cc:492] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Bootstrap starting.
I20260811 06:56:45.547359 19109 tablet_bootstrap.cc:654] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.552284 19111 tablet_bootstrap.cc:492] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: No bootstrap required, opened a new log
I20260811 06:56:45.552492 19111 ts_tablet_manager.cc:1397] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.009s
I20260811 06:56:45.553397 19111 raft_consensus.cc:348] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.553669 19111 raft_consensus.cc:374] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.553768 19111 raft_consensus.cc:729] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.554036 19111 consensus_queue.cc:260] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.556735 19111 ts_tablet_manager.cc:1428] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 06:56:45.560180 19109 tablet_bootstrap.cc:492] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: No bootstrap required, opened a new log
I20260811 06:56:45.560320 19109 ts_tablet_manager.cc:1397] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20260811 06:56:45.561055 19109 raft_consensus.cc:348] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.561318 19109 raft_consensus.cc:374] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.561403 19109 raft_consensus.cc:729] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22d6f250579411c820fde809c1a275c, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.561642 19109 consensus_queue.cc:260] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.562232 19109 ts_tablet_manager.cc:1428] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:56:45.696321 19078 raft_consensus.cc:482] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:45.696547 19078 raft_consensus.cc:504] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.697348 19078 leader_election.cc:288] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785)
I20260811 06:56:45.697934 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb5847d395b46d5b104ebbce449d29a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781" is_pre_election: true
I20260811 06:56:45.698061 18595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb5847d395b46d5b104ebbce449d29a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461ad0ff78f34b4887fe619ccad25d0e" is_pre_election: true
I20260811 06:56:45.698256 18345 raft_consensus.cc:2454] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 0.
I20260811 06:56:45.698336 18595 raft_consensus.cc:2454] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 0.
I20260811 06:56:45.698781 18718 leader_election.cc:302] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4acf00ed764e05b6ccfd9184370781, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:45.699055 19078 raft_consensus.cc:2781] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:45.699198 19078 raft_consensus.cc:482] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:45.699306 19078 raft_consensus.cc:3037] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.701543 19078 raft_consensus.cc:504] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.702116 19078 leader_election.cc:288] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 election: Requested vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785)
I20260811 06:56:45.702579 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb5847d395b46d5b104ebbce449d29a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781"
I20260811 06:56:45.702802 18345 raft_consensus.cc:3037] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.702780 18595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4fb5847d395b46d5b104ebbce449d29a" candidate_uuid: "d22d6f250579411c820fde809c1a275c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "461ad0ff78f34b4887fe619ccad25d0e"
I20260811 06:56:45.702972 18595 raft_consensus.cc:3037] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.705256 18345 raft_consensus.cc:2454] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 1.
I20260811 06:56:45.705639 18718 leader_election.cc:302] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4acf00ed764e05b6ccfd9184370781, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:45.705917 19078 raft_consensus.cc:2781] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:45.705932 18595 raft_consensus.cc:2454] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22d6f250579411c820fde809c1a275c in term 1.
I20260811 06:56:45.706095 19078 raft_consensus.cc:686] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 1 LEADER]: Becoming Leader. State: Replica: d22d6f250579411c820fde809c1a275c, State: Running, Role: LEADER
I20260811 06:56:45.706652 19078 consensus_queue.cc:237] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.709695 18065 catalog_manager.cc:5495] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c reported cstate change: term changed from 0 to 1, leader changed from <none> to d22d6f250579411c820fde809c1a275c (127.4.40.3). New cstate: current_term: 1 leader_uuid: "d22d6f250579411c820fde809c1a275c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } health_report { overall_health: HEALTHY } } }
I20260811 06:56:45.729521 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:45.724392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47140 (local address 127.4.40.3:41503)
0811 06:56:45.724602 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:45.724609 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:45.724711 (+ 102us) server_negotiation.cc:407] Connection header received
0811 06:56:45.724954 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:45.724958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:45.725030 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:45.725140 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:45.725509 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.726279 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.727621 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:45.728074 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:45.728295 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:45.728786 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:45.728897 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:45.728999 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:45.729155 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 06:56:45.729232 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:56:45.732225 18345 raft_consensus.cc:1261] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Refusing update from remote peer d22d6f250579411c820fde809c1a275c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:45.732271 18595 raft_consensus.cc:1261] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Refusing update from remote peer d22d6f250579411c820fde809c1a275c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:45.732909 18982 consensus_queue.cc:1035] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [LEADER]: Connected to new peer: Peer: permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:45.733230 19078 consensus_queue.cc:1035] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:45.744902 18061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:45.745189 18061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:45.747983 18061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Sending DeleteTablet for 3 replicas of tablet 4fb5847d395b46d5b104ebbce449d29a
I20260811 06:56:45.748842 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb5847d395b46d5b104ebbce449d29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:45 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59604
I20260811 06:56:45.748799 18305 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb5847d395b46d5b104ebbce449d29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:45 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:60824
I20260811 06:56:45.749152 19085 tablet_replica.cc:330] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: stopping tablet replica
I20260811 06:56:45.749173 18801 tablet_service.cc:1514] Processing DeleteTablet for tablet 4fb5847d395b46d5b104ebbce449d29a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:45 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:47114
I20260811 06:56:45.749387 19085 raft_consensus.cc:2229] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:45.749495 19089 tablet_replica.cc:330] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: stopping tablet replica
I20260811 06:56:45.749634 19085 raft_consensus.cc:2258] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:45.749709 19089 raft_consensus.cc:2229] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:45.749938 19089 raft_consensus.cc:2258] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:45.750205 19087 tablet_replica.cc:330] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: stopping tablet replica
I20260811 06:56:45.750098 18061 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 06:56:45.750391 19087 raft_consensus.cc:2229] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:45.750672 19087 raft_consensus.cc:2258] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 06:56:45.750739 18061 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:56:45.750810 19085 ts_tablet_manager.cc:1905] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:45.751605 19087 ts_tablet_manager.cc:1905] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:45.751735 19089 ts_tablet_manager.cc:1905] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:45.757339 19089 ts_tablet_manager.cc:1918] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:45.757493 19089 log.cc:1196] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/4fb5847d395b46d5b104ebbce449d29a
I20260811 06:56:45.757893 19089 ts_tablet_manager.cc:1939] T 4fb5847d395b46d5b104ebbce449d29a P 461ad0ff78f34b4887fe619ccad25d0e: Deleting consensus metadata
I20260811 06:56:45.758876 18041 catalog_manager.cc:4841] TS 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785): tablet 4fb5847d395b46d5b104ebbce449d29a (table test-table [id=24a5b6ed7b344b509053936be1701d09]) successfully deleted
I20260811 06:56:45.759712 19087 ts_tablet_manager.cc:1918] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:45.759958 19087 log.cc:1196] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/4fb5847d395b46d5b104ebbce449d29a
I20260811 06:56:45.760489 19087 ts_tablet_manager.cc:1939] T 4fb5847d395b46d5b104ebbce449d29a P d22d6f250579411c820fde809c1a275c: Deleting consensus metadata
I20260811 06:56:45.761605 18036 catalog_manager.cc:4841] TS d22d6f250579411c820fde809c1a275c (127.4.40.3:41503): tablet 4fb5847d395b46d5b104ebbce449d29a (table test-table [id=24a5b6ed7b344b509053936be1701d09]) successfully deleted
I20260811 06:56:45.762295 19085 ts_tablet_manager.cc:1918] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:56:45.762466 19085 log.cc:1196] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/4fb5847d395b46d5b104ebbce449d29a
I20260811 06:56:45.762889 19085 ts_tablet_manager.cc:1939] T 4fb5847d395b46d5b104ebbce449d29a P 3c4acf00ed764e05b6ccfd9184370781: Deleting consensus metadata
I20260811 06:56:45.763767 18041 catalog_manager.cc:4841] TS 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143): tablet 4fb5847d395b46d5b104ebbce449d29a (table test-table [id=24a5b6ed7b344b509053936be1701d09]) successfully deleted
I20260811 06:56:45.768019 18305 tablet_service.cc:1467] Processing CreateTablet for tablet d15fae41d1e6455ba182d2e346f83788 (DEFAULT_TABLE table=test-table [id=de592a1646d24f0ca4a09b4e0689d595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:45.768070 18546 tablet_service.cc:1467] Processing CreateTablet for tablet d15fae41d1e6455ba182d2e346f83788 (DEFAULT_TABLE table=test-table [id=de592a1646d24f0ca4a09b4e0689d595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:45.768491 18305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d15fae41d1e6455ba182d2e346f83788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.768525 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d15fae41d1e6455ba182d2e346f83788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.772428 18801 tablet_service.cc:1467] Processing CreateTablet for tablet d15fae41d1e6455ba182d2e346f83788 (DEFAULT_TABLE table=test-table [id=de592a1646d24f0ca4a09b4e0689d595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:56:45.772845 18801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d15fae41d1e6455ba182d2e346f83788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:45.773794 18970 tablet_bootstrap.cc:492] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Bootstrap starting.
I20260811 06:56:45.775156 18970 tablet_bootstrap.cc:654] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.777405 18970 tablet_bootstrap.cc:492] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: No bootstrap required, opened a new log
I20260811 06:56:45.777603 18970 ts_tablet_manager.cc:1397] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260811 06:56:45.777941 19109 tablet_bootstrap.cc:492] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Bootstrap starting.
I20260811 06:56:45.778471 18970 raft_consensus.cc:348] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.778743 18970 raft_consensus.cc:374] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.778812 18970 raft_consensus.cc:729] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4acf00ed764e05b6ccfd9184370781, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.779069 18970 consensus_queue.cc:260] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.779605 19109 tablet_bootstrap.cc:654] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.780776 18970 ts_tablet_manager.cc:1428] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260811 06:56:45.782054 19111 tablet_bootstrap.cc:492] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Bootstrap starting.
I20260811 06:56:45.783363 19109 tablet_bootstrap.cc:492] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: No bootstrap required, opened a new log
I20260811 06:56:45.783541 19109 ts_tablet_manager.cc:1397] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 06:56:45.784183 19111 tablet_bootstrap.cc:654] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:45.784235 19109 raft_consensus.cc:348] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.784503 19109 raft_consensus.cc:374] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.784611 19109 raft_consensus.cc:729] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22d6f250579411c820fde809c1a275c, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.784900 19109 consensus_queue.cc:260] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.786152 19109 ts_tablet_manager.cc:1428] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 06:56:45.793365 19111 tablet_bootstrap.cc:492] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: No bootstrap required, opened a new log
I20260811 06:56:45.793594 19111 ts_tablet_manager.cc:1397] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260811 06:56:45.794404 19111 raft_consensus.cc:348] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.794641 19111 raft_consensus.cc:374] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:45.794760 19111 raft_consensus.cc:729] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Initialized, Role: FOLLOWER
I20260811 06:56:45.794986 19111 consensus_queue.cc:260] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.795533 19111 ts_tablet_manager.cc:1428] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 06:56:45.956743 19066 raft_consensus.cc:482] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:56:45.956995 19066 raft_consensus.cc:504] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.957677 19066 leader_election.cc:288] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), d22d6f250579411c820fde809c1a275c (127.4.40.3:41503)
I20260811 06:56:45.958247 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d15fae41d1e6455ba182d2e346f83788" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781" is_pre_election: true
I20260811 06:56:45.958282 18847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d15fae41d1e6455ba182d2e346f83788" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22d6f250579411c820fde809c1a275c" is_pre_election: true
I20260811 06:56:45.958526 18345 raft_consensus.cc:2454] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 0.
I20260811 06:56:45.958549 18847 raft_consensus.cc:2454] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 0.
I20260811 06:56:45.958982 18465 leader_election.cc:302] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c4acf00ed764e05b6ccfd9184370781, 461ad0ff78f34b4887fe619ccad25d0e; no voters:
I20260811 06:56:45.959251 19066 raft_consensus.cc:2781] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:56:45.959372 19066 raft_consensus.cc:482] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:56:45.959506 19066 raft_consensus.cc:3037] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.961658 19066 raft_consensus.cc:504] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.962258 19066 leader_election.cc:288] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 election: Requested vote from peers 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143), d22d6f250579411c820fde809c1a275c (127.4.40.3:41503)
I20260811 06:56:45.962741 18345 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d15fae41d1e6455ba182d2e346f83788" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3c4acf00ed764e05b6ccfd9184370781"
I20260811 06:56:45.962811 18847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d15fae41d1e6455ba182d2e346f83788" candidate_uuid: "461ad0ff78f34b4887fe619ccad25d0e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22d6f250579411c820fde809c1a275c"
I20260811 06:56:45.962962 18345 raft_consensus.cc:3037] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.963007 18847 raft_consensus.cc:3037] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:45.965929 18847 raft_consensus.cc:2454] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 1.
I20260811 06:56:45.966071 18345 raft_consensus.cc:2454] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 461ad0ff78f34b4887fe619ccad25d0e in term 1.
I20260811 06:56:45.966374 18459 leader_election.cc:302] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461ad0ff78f34b4887fe619ccad25d0e, d22d6f250579411c820fde809c1a275c; no voters:
I20260811 06:56:45.966687 19066 raft_consensus.cc:2781] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:45.967051 19066 raft_consensus.cc:686] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 LEADER]: Becoming Leader. State: Replica: 461ad0ff78f34b4887fe619ccad25d0e, State: Running, Role: LEADER
I20260811 06:56:45.967374 19066 consensus_queue.cc:237] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } }
I20260811 06:56:45.970542 18061 catalog_manager.cc:5495] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e reported cstate change: term changed from 0 to 1, leader changed from <none> to 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2). New cstate: current_term: 1 leader_uuid: "461ad0ff78f34b4887fe619ccad25d0e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "461ad0ff78f34b4887fe619ccad25d0e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35785 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 } health_report { overall_health: UNKNOWN } } }
I20260811 06:56:46.031710 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:46.027045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59620 (local address 127.4.40.2:35785)
0811 06:56:46.027527 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:46.027531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:46.027547 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:56:46.027596 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:46.027599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:46.027657 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:46.027739 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:46.028127 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.028835 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.030080 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.030514 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.030602 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:46.030981 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:46.031081 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:46.031146 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:46.031283 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 06:56:46.031342 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":266,"threads_started":1}
I20260811 06:56:46.043803 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:46.039281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:59867 (local address 127.4.40.2:35785)
0811 06:56:46.039423 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:46.039427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:46.039880 (+ 453us) server_negotiation.cc:407] Connection header received
0811 06:56:46.040010 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:46.040013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:46.040070 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:46.040193 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:56:46.040560 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.041207 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.042447 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.042819 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.042909 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:46.043269 (+ 360us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:56:46.043352 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:56:46.043415 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:46.043547 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 06:56:46.043602 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 06:56:46.043845 19141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:46.039195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35785 (local address 127.4.40.2:59867)
0811 06:56:46.039794 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:46.039810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:46.039913 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:46.040204 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:46.040207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:46.040216 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 06:56:46.040426 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:46.040433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.041317 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.041320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:46.042328 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:46.042339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.042954 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.042958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:46.043095 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:46.043126 (+ 31us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 06:56:46.043434 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 06:56:46.043442 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:56:46.043502 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 06:56:46.043594 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":347,"threads_started":1}
I20260811 06:56:46.051105 18847 raft_consensus.cc:1261] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Refusing update from remote peer 461ad0ff78f34b4887fe619ccad25d0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:46.051120 18345 raft_consensus.cc:1261] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Refusing update from remote peer 461ad0ff78f34b4887fe619ccad25d0e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:56:46.051666 19131 consensus_queue.cc:1035] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22d6f250579411c820fde809c1a275c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 41503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:46.051940 19066 consensus_queue.cc:1035] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3c4acf00ed764e05b6ccfd9184370781" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:56:46.097797 18061 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:46.098088 18061 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:56:46.101166 18061 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6f429ed9ee142dfa9d943bc817e6872: Sending DeleteTablet for 3 replicas of tablet d15fae41d1e6455ba182d2e346f83788
I20260811 06:56:46.101866 18305 tablet_service.cc:1514] Processing DeleteTablet for tablet d15fae41d1e6455ba182d2e346f83788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:46 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:60824
I20260811 06:56:46.101936 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet d15fae41d1e6455ba182d2e346f83788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:46 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59604
I20260811 06:56:46.102001 18801 tablet_service.cc:1514] Processing DeleteTablet for tablet d15fae41d1e6455ba182d2e346f83788 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:56:46 UTC) from {username='oryx', principal='oryx/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:47114
I20260811 06:56:46.102171 19085 tablet_replica.cc:330] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: stopping tablet replica
I20260811 06:56:46.102288 19089 tablet_replica.cc:330] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: stopping tablet replica
I20260811 06:56:46.102288 19087 tablet_replica.cc:330] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: stopping tablet replica
I20260811 06:56:46.102368 19085 raft_consensus.cc:2229] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:46.102466 19089 raft_consensus.cc:2229] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:56:46.102463 19087 raft_consensus.cc:2229] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:56:46.102571 19085 raft_consensus.cc:2258] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:46.102643 19087 raft_consensus.cc:2258] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:46.102929 19089 raft_consensus.cc:2258] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:56:46.103649 19087 ts_tablet_manager.cc:1905] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:46.104082 19085 ts_tablet_manager.cc:1905] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:46.104125 19089 ts_tablet_manager.cc:1905] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:56:46.110455 19087 ts_tablet_manager.cc:1918] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:56:46.110589 19087 log.cc:1196] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/d15fae41d1e6455ba182d2e346f83788
I20260811 06:56:46.111056 19085 ts_tablet_manager.cc:1918] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:56:46.111096 19087 ts_tablet_manager.cc:1939] T d15fae41d1e6455ba182d2e346f83788 P d22d6f250579411c820fde809c1a275c: Deleting consensus metadata
I20260811 06:56:46.111238 19085 log.cc:1196] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/d15fae41d1e6455ba182d2e346f83788
I20260811 06:56:46.111464 19089 ts_tablet_manager.cc:1918] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 06:56:46.111667 19089 log.cc:1196] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/d15fae41d1e6455ba182d2e346f83788
I20260811 06:56:46.111740 19085 ts_tablet_manager.cc:1939] T d15fae41d1e6455ba182d2e346f83788 P 3c4acf00ed764e05b6ccfd9184370781: Deleting consensus metadata
I20260811 06:56:46.112145 18036 catalog_manager.cc:4841] TS d22d6f250579411c820fde809c1a275c (127.4.40.3:41503): tablet d15fae41d1e6455ba182d2e346f83788 (table test-table [id=de592a1646d24f0ca4a09b4e0689d595]) successfully deleted
I20260811 06:56:46.112247 19089 ts_tablet_manager.cc:1939] T d15fae41d1e6455ba182d2e346f83788 P 461ad0ff78f34b4887fe619ccad25d0e: Deleting consensus metadata
I20260811 06:56:46.113050 18041 catalog_manager.cc:4841] TS 3c4acf00ed764e05b6ccfd9184370781 (127.4.40.1:37143): tablet d15fae41d1e6455ba182d2e346f83788 (table test-table [id=de592a1646d24f0ca4a09b4e0689d595]) successfully deleted
I20260811 06:56:46.113555 18041 catalog_manager.cc:4841] TS 461ad0ff78f34b4887fe619ccad25d0e (127.4.40.2:35785): tablet d15fae41d1e6455ba182d2e346f83788 (table test-table [id=de592a1646d24f0ca4a09b4e0689d595]) successfully deleted
I20260811 06:56:46.114048 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 18184
Aug 11 06:56:46 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.1@KRBTEST.COM
I20260811 06:56:46.131315 19151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:46.115283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.4.40.1:37143)
0811 06:56:46.115785 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:46.115789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:46.115805 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:56:46.115847 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:46.115850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:46.115904 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:46.115989 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:46.117644 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.118336 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.120189 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.120607 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.120707 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:46.125479 (+ 4772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:46.125493 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:46.125508 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:46.125550 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:46.128494 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:46.129158 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:46.129165 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:46.129167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:46.129316 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:46.129732 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:46.129735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:46.129737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:46.130086 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:46.130417 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:46.130696 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 06:56:46.130913 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":316,"threads_started":1}
I20260811 06:56:46.132082 18410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:46.233249 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 18184
I20260811 06:56:46.260020 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 18425
Aug 11 06:56:46 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.2@KRBTEST.COM
I20260811 06:56:46.278394 19138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:46.260916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59632 (local address 127.4.40.2:35785)
0811 06:56:46.261101 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:46.261105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:46.261124 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:56:46.261324 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:46.261328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:46.261392 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:46.261477 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:46.263150 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.263915 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.265978 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.266453 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.266567 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:46.271948 (+ 5381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:46.271964 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:46.271980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:46.272021 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:46.275349 (+ 3328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:46.276055 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:46.276062 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:46.276064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:46.276323 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:46.276715 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:46.276720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:46.276724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:46.277107 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:46.277454 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:46.277771 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 06:56:46.277996 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 06:56:46.279134 18677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:46.431245 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 18425
I20260811 06:56:46.468645 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 18694
Aug 11 06:56:46 dist-test-slave-7vjv krb5kdc[17988](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431405, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.40.3@KRBTEST.COM
I20260811 06:56:46.483774 19154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:46.469435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.4.40.3:41503)
0811 06:56:46.469954 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:46.469958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:46.469975 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:56:46.470034 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:46.470037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:46.470102 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:46.470192 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:46.471676 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.472473 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.473681 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:46.474082 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:46.474177 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:46.478396 (+ 4219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:46.478412 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:46.478429 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:46.478462 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:46.481308 (+ 2846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:46.481926 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:46.481934 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:46.481936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:46.482075 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:46.482392 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:46.482395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:46.482397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:46.482737 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:46.482949 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:46.483196 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 06:56:46.483356 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":294,"threads_started":1}
I20260811 06:56:46.484630 18912 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:46.621331 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 18694
I20260811 06:56:46.650926 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 18009
I20260811 06:56:46.652241 18129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:46.769392 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 18009
2026-08-11T06:56:46Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6327 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:46 dist-test-slave-7vjv krb5kdc[19163](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:46 dist-test-slave-7vjv krb5kdc[19163](info): set up 2 sockets
Aug 11 06:56:46 dist-test-slave-7vjv krb5kdc[19163](info): commencing operation
krb5kdc: starting...
W20260811 06:56:48.149868 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.015s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:48 dist-test-slave-7vjv krb5kdc[19163](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:48.435226 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:38969
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:38969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:48.765399 19184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:48.765672 19184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:48.765793 19184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:48.772612 19184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:48.772722 19184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:48.772826 19184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:48.772923 19184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:48.772990 19184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:48.782485 19184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:38969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:38969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:48.783771 19184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:48.785115 19184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:48.797045 19196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:48.797163 19195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:48.797183 19199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:48.798439 19184 server_base.cc:1034] running on GCE node
I20260811 06:56:48.799337 19184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:48.800859 19184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:48.802074 19184 hybrid_clock.cc:648] HybridClock initialized: now 1786431408802027 us; error 74 us; skew 500 ppm
I20260811 06:56:48.802424 19184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T06:56:48Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2094 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:56:48 dist-test-slave-7vjv krb5kdc[19208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:56:48 dist-test-slave-7vjv krb5kdc[19208](info): set up 2 sockets
Aug 11 06:56:48 dist-test-slave-7vjv krb5kdc[19208](info): commencing operation
krb5kdc: starting...
W20260811 06:56:50.236805 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.002s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:56:50 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:56:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:56:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:56:50.515524 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36869
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:50.850908 19231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:50.851256 19231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:50.851445 19231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:50.858817 19231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:50.858925 19231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:50.859023 19231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:50.859136 19231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:50.859230 19231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:50.869575 19231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:50.870898 19231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:50.872224 19231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:50.883931 19241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:50.883993 19240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:50.884032 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:50.884752 19231 server_base.cc:1034] running on GCE node
I20260811 06:56:50.885574 19231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:50.887005 19231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:50.888191 19231 hybrid_clock.cc:648] HybridClock initialized: now 1786431410888148 us; error 36 us; skew 500 ppm
I20260811 06:56:50.888540 19231 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:50 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431410, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:50.896168 19231 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:50.897836 19231 webserver.cc:460] Webserver started at http://127.4.40.62:37397/ using document root <none> and password file <none>
I20260811 06:56:50.898576 19231 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:50.898728 19231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:50.899072 19231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:50.902261 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "4ed2be3d59294c578fd1c1cde6690fc9"
format_stamp: "Formatted at 2026-08-11 06:56:50 on dist-test-slave-7vjv"
server_key: "848aa91173e2f60e01cc6808ec641618"
server_key_iv: "bfe16052da1c4ab844d8fa0c84be0f5d"
server_key_version: "encryptionkey@0"
I20260811 06:56:50.903120 19231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "4ed2be3d59294c578fd1c1cde6690fc9"
format_stamp: "Formatted at 2026-08-11 06:56:50 on dist-test-slave-7vjv"
server_key: "848aa91173e2f60e01cc6808ec641618"
server_key_iv: "bfe16052da1c4ab844d8fa0c84be0f5d"
server_key_version: "encryptionkey@0"
I20260811 06:56:50.908326 19231 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:56:50.911612 19257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:50.912896 19231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:50.913084 19231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "4ed2be3d59294c578fd1c1cde6690fc9"
format_stamp: "Formatted at 2026-08-11 06:56:50 on dist-test-slave-7vjv"
server_key: "848aa91173e2f60e01cc6808ec641618"
server_key_iv: "bfe16052da1c4ab844d8fa0c84be0f5d"
server_key_version: "encryptionkey@0"
I20260811 06:56:50.913270 19231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:50.932969 19231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:50.936831 19231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:50.937129 19231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:50.956444 19231 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:36869
I20260811 06:56:50.956485 19335 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:36869 every 8 connection(s)
I20260811 06:56:50.957891 19231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:50.961740 19336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:56:50.963392 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 19231
I20260811 06:56:50.963622 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:56:50.964363 4256 external_mini_cluster.cc:1428] Setting key aea0833b59c8dc242be64222c64e3c32
I20260811 06:56:50.974012 19336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Bootstrap starting.
I20260811 06:56:50.977911 19336 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Neither blocks nor log segments found. Creating new log.
I20260811 06:56:50.979187 19336 log.cc:824] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:56:50 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:50.982726 19336 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: No bootstrap required, opened a new log
I20260811 06:56:50.988942 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:50.966452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53014 (local address 127.4.40.62:36869)
0811 06:56:50.967338 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:50.967350 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:50.967401 (+ 51us) server_negotiation.cc:407] Connection header received
0811 06:56:50.969132 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:50.969153 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:50.969526 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:50.969816 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:50.971654 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:50.973040 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:50.974734 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:50.975363 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:50.975499 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:50.980663 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:50.980676 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:50.980693 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:50.980737 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:50.984632 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:50.985279 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:50.985286 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:50.985294 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:50.985505 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:50.985842 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:50.985847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:50.985850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:50.986559 (+ 709us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:50.986862 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:50.987171 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 06:56:50.987572 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":399,"threads_started":1}
I20260811 06:56:50.992044 19336 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:50.992427 19336 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:56:50.992501 19336 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed2be3d59294c578fd1c1cde6690fc9, State: Initialized, Role: FOLLOWER
I20260811 06:56:50.992933 19336 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:50.993137 19336 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:50.993229 19336 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:50.993348 19336 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:56:50.995361 19336 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:50.995780 19336 leader_election.cc:302] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ed2be3d59294c578fd1c1cde6690fc9; no voters:
I20260811 06:56:50.996436 19336 leader_election.cc:288] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:56:50.996546 19344 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:56:50.997396 19344 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 LEADER]: Becoming Leader. State: Replica: 4ed2be3d59294c578fd1c1cde6690fc9, State: Running, Role: LEADER
I20260811 06:56:50.997910 19344 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:50.998490 19336 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:51.000672 19347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } } }
I20260811 06:56:51.001003 19347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:51.002614 19360 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:51.002653 19349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ed2be3d59294c578fd1c1cde6690fc9. Latest consensus state: current_term: 1 leader_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } } }
I20260811 06:56:51.002948 19349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:51.008304 19360 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:51.017159 19360 catalog_manager.cc:1347] Generated new cluster ID: 176e1e33f6634ceea26a883976f02ec3
I20260811 06:56:51.017381 19360 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:51.038015 19360 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:56:51.039342 19360 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:51.047385 19360 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Generated new TSK 0
I20260811 06:56:51.048734 19360 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:56:51.122982 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36869
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:51.476723 19388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:51.477003 19388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:51.477214 19388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:51.484851 19388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:51.484992 19388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:51.485193 19388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:51.495760 19388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:51.497135 19388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:51.498950 19388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:51.513454 19394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:51.513521 19397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:51.513497 19395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:51.514473 19388 server_base.cc:1034] running on GCE node
I20260811 06:56:51.515055 19388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:51.516000 19388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:51.517233 19388 hybrid_clock.cc:648] HybridClock initialized: now 1786431411517182 us; error 50 us; skew 500 ppm
I20260811 06:56:51.517580 19388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:51 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431411, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:51.520992 19388 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:56:51.522459 19388 webserver.cc:460] Webserver started at http://127.4.40.1:38825/ using document root <none> and password file <none>
I20260811 06:56:51.523135 19388 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:51.523252 19388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:51.523567 19388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:51.526682 19388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "af5c65fd6e2b4ce1a5933799ccd2e39a"
format_stamp: "Formatted at 2026-08-11 06:56:51 on dist-test-slave-7vjv"
server_key: "fb7ad3921bcb82562d943764d7cca761"
server_key_iv: "69c6fbad65362cd34648e795ad8a635b"
server_key_version: "encryptionkey@0"
I20260811 06:56:51.527557 19388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "af5c65fd6e2b4ce1a5933799ccd2e39a"
format_stamp: "Formatted at 2026-08-11 06:56:51 on dist-test-slave-7vjv"
server_key: "fb7ad3921bcb82562d943764d7cca761"
server_key_iv: "69c6fbad65362cd34648e795ad8a635b"
server_key_version: "encryptionkey@0"
I20260811 06:56:51.533171 19388 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 06:56:51.536666 19405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:51.537989 19388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:56:51.538195 19388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "af5c65fd6e2b4ce1a5933799ccd2e39a"
format_stamp: "Formatted at 2026-08-11 06:56:51 on dist-test-slave-7vjv"
server_key: "fb7ad3921bcb82562d943764d7cca761"
server_key_iv: "69c6fbad65362cd34648e795ad8a635b"
server_key_version: "encryptionkey@0"
I20260811 06:56:51.538409 19388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:51.552249 19388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:51.556228 19388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:51.556535 19388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:51.557633 19388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:51.559139 19388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:51.559257 19388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:51.559422 19388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:51.559517 19388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:51.595732 19388 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:35375
I20260811 06:56:51.595778 19591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:35375 every 8 connection(s)
I20260811 06:56:51.597296 19388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:51.601506 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 19388
I20260811 06:56:51.601763 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:56:51.602643 4256 external_mini_cluster.cc:1428] Setting key d150f9b831e1a87c07be1d4efde68d4b
Aug 11 06:56:51 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431411, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:51.622108 19596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:51.600380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:58719 (local address 127.4.40.62:36869)
0811 06:56:51.600921 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:51.600926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:51.602250 (+ 1324us) server_negotiation.cc:407] Connection header received
0811 06:56:51.604954 (+ 2704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:51.604961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:51.605061 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:51.605223 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:51.606985 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:51.607901 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:51.609810 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:51.610302 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:51.610495 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:51.614846 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:51.614852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:51.614856 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:51.614895 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:51.618393 (+ 3498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:51.619258 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:51.619266 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:51.619272 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:51.619426 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:51.619968 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:51.619976 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:51.619983 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:51.620392 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:51.620548 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:51.621362 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 06:56:51.621621 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":289,"threads_started":1}
I20260811 06:56:51.622998 19597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:51.600879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:58719)
0811 06:56:51.602050 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:51.602114 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:51.604594 (+ 2480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:51.605502 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:51.605512 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:51.606134 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:51.606770 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:51.606790 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:51.608146 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:51.608153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:51.609617 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:51.609632 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:51.610603 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:51.610614 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:51.610921 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:51.611777 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:51.611806 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:51.614576 (+ 2770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:51.618555 (+ 3979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:51.618563 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:51.618577 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:51.619104 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:51.619547 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:51.619550 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:51.619553 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:51.619783 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:51.620539 (+ 756us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:51.620546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:51.620938 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 06:56:51.621204 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 06:56:51.621645 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":798,"thread_start_us":459,"threads_started":1}
I20260811 06:56:51.624795 19594 heartbeater.cc:347] Connected to a master server at 127.4.40.62:36869
I20260811 06:56:51.625125 19594 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:51.626067 19594 heartbeater.cc:511] Master 127.4.40.62:36869 requested a full tablet report, sending...
I20260811 06:56:51.628669 19287 ts_manager.cc:194] Registered new tserver with Master: af5c65fd6e2b4ce1a5933799ccd2e39a (127.4.40.1:35375)
I20260811 06:56:51.630939 19287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:58719
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:56:51.671149 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36869
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:52.004494 19603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:52.004791 19603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:52.004913 19603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:52.012115 19603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:52.012214 19603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:52.012375 19603 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:52.022110 19603 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:52.023265 19603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:52.024600 19603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:52.037498 19609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:52.037637 19613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:52.037665 19610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:52.038610 19603 server_base.cc:1034] running on GCE node
I20260811 06:56:52.039198 19603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:52.040211 19603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:52.041442 19603 hybrid_clock.cc:648] HybridClock initialized: now 1786431412041376 us; error 53 us; skew 500 ppm
I20260811 06:56:52.041775 19603 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:52 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431412, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:52.045173 19603 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:56:52.046566 19603 webserver.cc:460] Webserver started at http://127.4.40.2:34551/ using document root <none> and password file <none>
I20260811 06:56:52.047222 19603 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:52.047322 19603 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:52.047588 19603 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:52.050443 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "e37c2f34d2cf466b81f34c10ad139379"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "18f7b61b8c26e284bac509bbdd700aba"
server_key_iv: "5d68772428bfb8bfae94f8fb7831747e"
server_key_version: "encryptionkey@0"
I20260811 06:56:52.051234 19603 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "e37c2f34d2cf466b81f34c10ad139379"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "18f7b61b8c26e284bac509bbdd700aba"
server_key_iv: "5d68772428bfb8bfae94f8fb7831747e"
server_key_version: "encryptionkey@0"
I20260811 06:56:52.056361 19603 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:56:52.059692 19629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:52.060974 19603 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:52.061156 19603 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "e37c2f34d2cf466b81f34c10ad139379"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "18f7b61b8c26e284bac509bbdd700aba"
server_key_iv: "5d68772428bfb8bfae94f8fb7831747e"
server_key_version: "encryptionkey@0"
I20260811 06:56:52.061347 19603 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:52.080060 19603 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:52.084424 19603 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:52.084718 19603 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:52.085650 19603 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:52.087003 19603 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:52.087109 19603 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:52.087262 19603 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:52.087335 19603 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:52.121068 19603 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:40255
I20260811 06:56:52.121155 19839 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:40255 every 8 connection(s)
I20260811 06:56:52.122363 19603 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:52.127231 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 19603
I20260811 06:56:52.127413 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:56:52.127838 4256 external_mini_cluster.cc:1428] Setting key 32dd9c31a60cc8ae90ef2391f75a2090
Aug 11 06:56:52 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431412, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:52.145236 19844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:52.124848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:41221 (local address 127.4.40.62:36869)
0811 06:56:52.125308 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:52.125314 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:52.126266 (+ 952us) server_negotiation.cc:407] Connection header received
0811 06:56:52.128148 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:52.128154 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:52.128240 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:52.128368 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:52.130285 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.131388 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.132959 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.133675 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.133817 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:52.137904 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:52.137913 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:52.137921 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:52.137968 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:52.141651 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:52.142419 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:52.142427 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:52.142433 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:52.142628 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:52.143150 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:52.143154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:52.143156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:52.143405 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:52.143530 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:52.144469 (+ 939us) server_negotiation.cc:299] Negotiation successful
0811 06:56:52.144729 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":220,"threads_started":1}
I20260811 06:56:52.146008 19845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:52.125416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:41221)
0811 06:56:52.126107 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:52.126150 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:52.127834 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:52.128609 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:52.128620 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:52.129315 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:52.130119 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:52.130138 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.131527 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.131530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:52.132813 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:52.132825 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.133921 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.133927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:52.134137 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:52.135019 (+ 882us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:52.135056 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:52.137673 (+ 2617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:52.141787 (+ 4114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:52.141793 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:52.141804 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:52.142305 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:52.142760 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:52.142766 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:52.142770 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:52.143026 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:52.143537 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:52.143547 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:52.143983 (+ 436us) client_negotiation.cc:769] Sending connection context
0811 06:56:52.144373 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 06:56:52.144779 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":268,"threads_started":1}
I20260811 06:56:52.147495 19840 heartbeater.cc:347] Connected to a master server at 127.4.40.62:36869
I20260811 06:56:52.147799 19840 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:52.148586 19840 heartbeater.cc:511] Master 127.4.40.62:36869 requested a full tablet report, sending...
I20260811 06:56:52.150100 19287 ts_manager.cc:194] Registered new tserver with Master: e37c2f34d2cf466b81f34c10ad139379 (127.4.40.2:40255)
I20260811 06:56:52.151350 19287 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:41221
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:56:52.191016 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36869
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:52.532959 19851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:52.533242 19851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:52.533356 19851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:52.540810 19851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:52.540931 19851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:52.541064 19851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:52.550860 19851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:52.552084 19851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:52.553685 19851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:52.567755 19860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:52.567798 19864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:52.567826 19859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:52.568356 19851 server_base.cc:1034] running on GCE node
I20260811 06:56:52.569033 19851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:52.569912 19851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:52.571130 19851 hybrid_clock.cc:648] HybridClock initialized: now 1786431412571088 us; error 29 us; skew 500 ppm
I20260811 06:56:52.571462 19851 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:52 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431412, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:52.574620 19851 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:56:52.576061 19851 webserver.cc:460] Webserver started at http://127.4.40.3:43423/ using document root <none> and password file <none>
I20260811 06:56:52.576787 19851 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:52.576906 19851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:52.577159 19851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:56:52.579929 19851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "f931796ff75f4c7aaffdd311bd273b63"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "47f27e004ca4e36b189dd9adec727b25"
server_key_iv: "5bff39a9a71ed3dacc5ec129e7fc6d83"
server_key_version: "encryptionkey@0"
I20260811 06:56:52.580953 19851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "f931796ff75f4c7aaffdd311bd273b63"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "47f27e004ca4e36b189dd9adec727b25"
server_key_iv: "5bff39a9a71ed3dacc5ec129e7fc6d83"
server_key_version: "encryptionkey@0"
I20260811 06:56:52.586159 19851 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:56:52.589519 19875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:52.590765 19851 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:56:52.590950 19851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "f931796ff75f4c7aaffdd311bd273b63"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "47f27e004ca4e36b189dd9adec727b25"
server_key_iv: "5bff39a9a71ed3dacc5ec129e7fc6d83"
server_key_version: "encryptionkey@0"
I20260811 06:56:52.591159 19851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:52.623286 19851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:52.629693 19851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:52.630041 19851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:52.631206 19851 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:52.632992 19851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:52.633109 19851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:52.633239 19851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:52.633313 19851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:52.634846 19594 heartbeater.cc:503] Master 127.4.40.62:36869 was elected leader, sending a full tablet report...
I20260811 06:56:52.671864 19851 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:39863
I20260811 06:56:52.671940 20070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:39863 every 8 connection(s)
I20260811 06:56:52.673173 19851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:52.679311 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 19851
I20260811 06:56:52.679514 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:56:52.679968 4256 external_mini_cluster.cc:1428] Setting key 6dd8542a668ec94132b7f387c658510f
Aug 11 06:56:52 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431412, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:56:52.693192 20075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:52.675711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:37721 (local address 127.4.40.62:36869)
0811 06:56:52.676245 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:52.676250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:52.677112 (+ 862us) server_negotiation.cc:407] Connection header received
0811 06:56:52.679000 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:52.679006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:52.679114 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:52.679240 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:52.680995 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.682042 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.683653 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.684040 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.684232 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:52.687585 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:52.687591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:52.687594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:52.687625 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:52.690047 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:52.690769 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:52.690776 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:52.690780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:52.690907 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:52.691341 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:52.691358 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:52.691362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:52.691625 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:52.691793 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:52.692577 (+ 784us) server_negotiation.cc:299] Negotiation successful
0811 06:56:52.692800 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":268,"threads_started":1}
I20260811 06:56:52.694201 20076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:52.676231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:37721)
0811 06:56:52.676943 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:52.676997 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:52.678713 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:52.679508 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:52.679521 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:52.680088 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:52.680807 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:52.680829 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.682195 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.682204 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:52.683514 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:52.683526 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.684317 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.684324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:52.684595 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:52.685225 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:52.685246 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:52.687369 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:56:52.690184 (+ 2815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:52.690192 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:52.690207 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:52.690649 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:52.691032 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:56:52.691038 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:56:52.691042 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:56:52.691224 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:56:52.691769 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:56:52.691793 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:56:52.692161 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 06:56:52.692492 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 06:56:52.692974 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":283,"threads_started":1}
I20260811 06:56:52.695688 20072 heartbeater.cc:347] Connected to a master server at 127.4.40.62:36869
I20260811 06:56:52.695991 20072 heartbeater.cc:464] Registering TS with master...
I20260811 06:56:52.696759 20072 heartbeater.cc:511] Master 127.4.40.62:36869 requested a full tablet report, sending...
I20260811 06:56:52.698221 19286 ts_manager.cc:194] Registered new tserver with Master: f931796ff75f4c7aaffdd311bd273b63 (127.4.40.3:39863)
I20260811 06:56:52.699204 19286 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:37721
I20260811 06:56:52.707512 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab.
Entry for principal oryx/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab.
WARNING: no policy specified for oryx/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab.
Entry for principal oryx/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab.
WARNING: no policy specified for oryx/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab.
Entry for principal oryx/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab.
I20260811 06:56:52.874835 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 19388
Aug 11 06:56:52 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:56:52.891242 20089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:52.875986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.4.40.1:35375)
0811 06:56:52.876708 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:52.876715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:52.876742 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:56:52.876806 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:52.876812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:52.877032 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:52.877192 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:52.878552 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.879593 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.880851 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:52.881418 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:52.881517 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:52.885487 (+ 3970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:52.885499 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:52.885513 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:52.885549 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:52.888308 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:52.889047 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:52.889054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:52.889056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:52.889187 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:52.889581 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:52.889585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:52.889587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:52.889930 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:52.890258 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:52.890571 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 06:56:52.890845 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":376,"threads_started":1}
I20260811 06:56:52.892553 19585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:52.976697 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 19388
I20260811 06:56:52.997421 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 19603
Aug 11 06:56:53 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:56:53.014241 20095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:52.998389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38556 (local address 127.4.40.2:40255)
0811 06:56:52.999049 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:52.999056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:52.999116 (+ 60us) server_negotiation.cc:407] Connection header received
0811 06:56:52.999221 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:52.999232 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:52.999447 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:52.999597 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:53.001019 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:53.002026 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:53.003224 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:53.003841 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:53.003941 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:53.008363 (+ 4422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:53.008374 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:53.008388 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:53.008426 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:53.011221 (+ 2795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:53.011923 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:53.011930 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:53.011933 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:53.012066 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:53.012554 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:53.012561 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:53.012564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:53.012918 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:53.013266 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:53.013567 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 06:56:53.013873 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":330,"threads_started":1}
I20260811 06:56:53.015547 19827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:53.098091 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 19603
I20260811 06:56:53.122035 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 19851
Aug 11 06:56:53 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431410, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:56:53.138939 20098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:53.123147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47372 (local address 127.4.40.3:39863)
0811 06:56:53.123669 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:53.123676 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:53.123706 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:56:53.123765 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:53.123771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:53.123989 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:53.124174 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:53.125513 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:53.126922 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:53.128427 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:53.128989 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:53.129089 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:53.133694 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:53.133705 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:53.133719 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:53.133749 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:53.136490 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:53.137075 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:53.137082 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:53.137084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:53.137194 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:53.137507 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:53.137510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:53.137511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:53.137866 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:53.138091 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:53.138341 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 06:56:53.138595 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":257,"threads_started":1}
I20260811 06:56:53.140290 20064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:53.224138 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 19851
I20260811 06:56:53.253038 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 19231
I20260811 06:56:53.254328 19331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:56:53.358836 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 19231
I20260811 06:56:53.391762 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36869
--webserver_interface=127.4.40.62
--webserver_port=37397
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36869
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:56:53.727131 20102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:53.727444 20102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:53.727614 20102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:53.734539 20102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:56:53.734673 20102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:53.734781 20102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:53.734879 20102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:56:53.734967 20102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:56:53.744516 20102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=37397
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:53.745780 20102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:53.747097 20102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:53.758914 20115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:53.758890 20113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:53.759016 20111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:53.760177 20102 server_base.cc:1034] running on GCE node
I20260811 06:56:53.760969 20102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:53.762269 20102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:53.763470 20102 hybrid_clock.cc:648] HybridClock initialized: now 1786431413763423 us; error 39 us; skew 500 ppm
I20260811 06:56:53.763764 20102 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:53 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431413, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:53.766832 20102 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:56:53.768313 20102 webserver.cc:460] Webserver started at http://127.4.40.62:37397/ using document root <none> and password file <none>
I20260811 06:56:53.768983 20102 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:53.769148 20102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:53.776091 20102 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 06:56:53.778898 20128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:53.780285 20102 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:56:53.780478 20102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "4ed2be3d59294c578fd1c1cde6690fc9"
format_stamp: "Formatted at 2026-08-11 06:56:50 on dist-test-slave-7vjv"
server_key: "848aa91173e2f60e01cc6808ec641618"
server_key_iv: "bfe16052da1c4ab844d8fa0c84be0f5d"
server_key_version: "encryptionkey@0"
I20260811 06:56:53.781426 20102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:53.806924 20102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:53.810840 20102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:53.811131 20102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:53.830544 20102 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:36869
I20260811 06:56:53.830592 20210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:36869 every 8 connection(s)
I20260811 06:56:53.832136 20102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:56:53.838894 20212 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:56:53.841549 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 20102
I20260811 06:56:53.842234 20212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Bootstrap starting.
I20260811 06:56:53.842422 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:35375
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:36869
--webserver_port=38825
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:56:53.863526 20212 log.cc:824] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:56:53.875977 20212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:56:53.876739 20212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Bootstrap complete.
I20260811 06:56:53.891038 20212 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:53.892306 20212 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ed2be3d59294c578fd1c1cde6690fc9, State: Initialized, Role: FOLLOWER
I20260811 06:56:53.893043 20212 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:53.893311 20212 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:56:53.893421 20212 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:56:53.893627 20212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:56:53.899137 20212 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:53.899731 20212 leader_election.cc:302] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ed2be3d59294c578fd1c1cde6690fc9; no voters:
I20260811 06:56:53.900386 20212 leader_election.cc:288] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:56:53.900522 20219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:56:53.901844 20219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [term 2 LEADER]: Becoming Leader. State: Replica: 4ed2be3d59294c578fd1c1cde6690fc9, State: Running, Role: LEADER
I20260811 06:56:53.902336 20219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } }
I20260811 06:56:53.902532 20212 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:56:53.904727 20221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ed2be3d59294c578fd1c1cde6690fc9. Latest consensus state: current_term: 2 leader_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } } }
I20260811 06:56:53.905125 20221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:53.905706 20220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ed2be3d59294c578fd1c1cde6690fc9" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36869 } } }
I20260811 06:56:53.906065 20220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9 [sys.catalog]: This master's current role is: LEADER
I20260811 06:56:53.908257 20228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:56:53.917503 20228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:56:53.918998 20228 catalog_manager.cc:1259] Loaded cluster ID: 176e1e33f6634ceea26a883976f02ec3
I20260811 06:56:53.919148 20228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:56:53.923183 20228 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:56:53.924751 20228 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4ed2be3d59294c578fd1c1cde6690fc9: Loaded TSK: 0
I20260811 06:56:53.926330 20228 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:56:54.214937 20214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:54.215219 20214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:54.215382 20214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:54.222738 20214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:54.222855 20214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:54.223023 20214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:56:54.233855 20214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.1.keytab
--principal=oryx/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:35375
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=38825
--tserver_master_addrs=127.4.40.62:36869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:54.235175 20214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:54.236600 20214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:54.252303 20258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:54.252746 20264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:54.252786 20254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:54.253870 20214 server_base.cc:1034] running on GCE node
I20260811 06:56:54.254616 20214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:54.255618 20214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:54.256891 20214 hybrid_clock.cc:648] HybridClock initialized: now 1786431414256828 us; error 51 us; skew 500 ppm
I20260811 06:56:54.257241 20214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431414, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:54.261821 20214 init.cc:377] Logged in from keytab as oryx/127.4.40.1@KRBTEST.COM (short username oryx)
I20260811 06:56:54.264654 20214 webserver.cc:460] Webserver started at http://127.4.40.1:38825/ using document root <none> and password file <none>
I20260811 06:56:54.265413 20214 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:54.265581 20214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:54.271560 20214 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 06:56:54.274590 20277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:54.276213 20214 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:56:54.276413 20214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "af5c65fd6e2b4ce1a5933799ccd2e39a"
format_stamp: "Formatted at 2026-08-11 06:56:51 on dist-test-slave-7vjv"
server_key: "fb7ad3921bcb82562d943764d7cca761"
server_key_iv: "69c6fbad65362cd34648e795ad8a635b"
server_key_version: "encryptionkey@0"
I20260811 06:56:54.277426 20214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:54.295135 20214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:54.299896 20214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:54.300269 20214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:54.301297 20214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:54.302796 20214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:54.302919 20214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:54.303066 20214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:54.303179 20214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:54.351068 20214 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:35375
I20260811 06:56:54.351182 20458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:35375 every 8 connection(s)
I20260811 06:56:54.353055 20214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:56:54.358187 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 20214
I20260811 06:56:54.358865 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:40255
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:36869
--webserver_port=34551
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:54.378633 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:54.356875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:38425)
0811 06:56:54.358076 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:54.358147 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:54.361348 (+ 3201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:54.364219 (+ 2871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:54.364227 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:54.365178 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:54.366042 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.366074 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.368462 (+ 2388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.368478 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.370916 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.370935 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.372774 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.372782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.373081 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.373899 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:54.373929 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:54.377707 (+ 3778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":624,"threads_started":1}
W20260811 06:56:54.379331 20460 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:36869 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.40.62:36869: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
W20260811 06:56:54.380395 20463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:54.356797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:38425 (local address 127.4.40.62:36869)
0811 06:56:54.357641 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:54.357661 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:54.358359 (+ 698us) server_negotiation.cc:407] Connection header received
0811 06:56:54.362212 (+ 3853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:54.362242 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:54.362821 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:54.363299 (+ 478us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:54.366281 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.368276 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.371124 (+ 2848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.372524 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.372784 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.378195 (+ 5411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:38425: BlockingRecv error: recv got EOF from 127.4.40.1:38425 (error 108)
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":290,"threads_started":1}
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:54.389027 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:54.380524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:52111)
0811 06:56:54.380759 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:54.380790 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:54.380888 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:54.381383 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:54.381388 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:54.381806 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:54.381989 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.382004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.383229 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.383237 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.384278 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.384290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.384997 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.385003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.385203 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.385793 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:54.385828 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:54.388742 (+ 2914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131}
W20260811 06:56:54.389181 20463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:54.380694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:52111 (local address 127.4.40.62:36869)
0811 06:56:54.380959 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:54.380964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:54.380983 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:56:54.381036 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:54.381042 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:54.381114 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:54.381205 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:54.382166 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.383088 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.384409 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.384803 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.384955 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.388928 (+ 3973us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:52111: BlockingRecv error: recv got EOF from 127.4.40.1:52111 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":2646,"threads_started":1}
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:54.398619 20474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:54.389928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:56257)
0811 06:56:54.390203 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:54.390223 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:54.390319 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:54.390781 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:54.390784 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:54.391054 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:54.391193 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.391200 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.392871 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.392874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.393757 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.393767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.394885 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.394889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.395016 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.395643 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:54.395659 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:54.398424 (+ 2765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170}
W20260811 06:56:54.398911 20460 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:56:54.399253 20463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:54.390194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:56257 (local address 127.4.40.62:36869)
0811 06:56:54.390415 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:54.390425 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:54.390447 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:56:54.390521 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:54.390529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:54.390639 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:54.390769 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:54.391340 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.392730 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.393910 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.394739 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.394902 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.398691 (+ 3789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:56257: BlockingRecv error: recv got EOF from 127.4.40.1:56257 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 06:56:54.706319 20476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:54.706588 20476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:54.706683 20476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:54.713850 20476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:54.713956 20476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:54.714092 20476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:56:54.723344 20476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.2.keytab
--principal=oryx/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:40255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=34551
--tserver_master_addrs=127.4.40.62:36869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:54.724584 20476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:54.725864 20476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:54.739125 20486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:54.739213 20487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:54.739233 20493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:54.740338 20476 server_base.cc:1034] running on GCE node
I20260811 06:56:54.740942 20476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:54.741815 20476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:54.743042 20476 hybrid_clock.cc:648] HybridClock initialized: now 1786431414743002 us; error 53 us; skew 500 ppm
I20260811 06:56:54.743347 20476 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431414, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:54.746583 20476 init.cc:377] Logged in from keytab as oryx/127.4.40.2@KRBTEST.COM (short username oryx)
I20260811 06:56:54.748049 20476 webserver.cc:460] Webserver started at http://127.4.40.2:34551/ using document root <none> and password file <none>
I20260811 06:56:54.748761 20476 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:54.748934 20476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:54.754351 20476 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 06:56:54.757104 20500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:54.758455 20476 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:56:54.758662 20476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "e37c2f34d2cf466b81f34c10ad139379"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "18f7b61b8c26e284bac509bbdd700aba"
server_key_iv: "5d68772428bfb8bfae94f8fb7831747e"
server_key_version: "encryptionkey@0"
I20260811 06:56:54.759675 20476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:54.776259 20476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:54.780678 20476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:54.780952 20476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:54.781988 20476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:54.783427 20476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:54.783564 20476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:54.783691 20476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:54.783778 20476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:54.818289 20476 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:40255
I20260811 06:56:54.818385 20699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:40255 every 8 connection(s)
I20260811 06:56:54.819768 20476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:56:54.825078 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 20476
I20260811 06:56:54.825886 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:39863
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:36869
--webserver_port=43423
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 06:56:54.838676 20463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:54.822455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:44051 (local address 127.4.40.62:36869)
0811 06:56:54.822642 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:54.822646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:54.823735 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 06:56:54.826344 (+ 2609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:54.826349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:54.826418 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:54.826500 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:54.828634 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.829370 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.831817 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.832561 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.832694 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.838417 (+ 5723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:44051: BlockingRecv error: recv got EOF from 127.4.40.2:44051 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:56:54.839746 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:54.822887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:44051)
0811 06:56:54.823580 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:54.823624 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:54.826021 (+ 2397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:54.826764 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:54.826777 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:54.827591 (+ 814us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:54.828447 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.828474 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.829553 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.829561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.831655 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.831672 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.832803 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.832813 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.833161 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.833973 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:54.834009 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:54.838225 (+ 4216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":275,"threads_started":1}
W20260811 06:56:54.840698 20700 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:36869 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.40.62:36869: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:54.849707 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:54.841273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:58225)
0811 06:56:54.841448 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:54.841468 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:54.841560 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:54.842100 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:54.842104 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:54.842481 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:54.842653 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.842663 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.844180 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.844184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.845073 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.845084 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.845768 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.845774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.845938 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.846529 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:54.846546 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:54.849429 (+ 2883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:56:54.849874 20463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:54.841587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:58225 (local address 127.4.40.62:36869)
0811 06:56:54.841779 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:54.841787 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:54.841806 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:56:54.841858 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:54.841865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:54.841941 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:54.842051 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:54.842821 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.843993 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.845213 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.845615 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.845756 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.849641 (+ 3885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:58225: BlockingRecv error: recv got EOF from 127.4.40.2:58225 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:54 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:54.858114 20702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:54.850575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:38885)
0811 06:56:54.850750 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:54.850769 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:54.850860 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:54.851205 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:54.851210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:54.851582 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:54.851750 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.851758 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.852800 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.852806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.853672 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:54.853682 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.854322 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.854327 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:54.854497 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.855063 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:54.855078 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:54.857917 (+ 2839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 06:56:54.858461 20700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:56:54.858870 20463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:54.850684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:38885 (local address 127.4.40.62:36869)
0811 06:56:54.850838 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:54.850843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:54.850859 (+ 16us) server_negotiation.cc:407] Connection header received
0811 06:56:54.850999 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:54.851003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:54.851066 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:54.851180 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:54.851885 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.852668 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.853805 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:54.854184 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:54.854312 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:54.858607 (+ 4295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:38885: BlockingRecv error: recv got EOF from 127.4.40.2:38885 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 06:56:55.166963 20703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:56:55.167279 20703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:56:55.167446 20703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:56:55.174788 20703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:56:55.174880 20703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:56:55.174998 20703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:56:55.184661 20703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/krb5kdc/oryx_127.4.40.3.keytab
--principal=oryx/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:39863
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=43423
--tserver_master_addrs=127.4.40.62:36869
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:56:55.185904 20703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:56:55.187121 20703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:56:55.200445 20711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:55.200551 20714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:56:55.200588 20712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:56:55.201346 20703 server_base.cc:1034] running on GCE node
I20260811 06:56:55.201893 20703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:56:55.202840 20703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:56:55.204082 20703 hybrid_clock.cc:648] HybridClock initialized: now 1786431415204010 us; error 53 us; skew 500 ppm
I20260811 06:56:55.204456 20703 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431415, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:56:55.207970 20703 init.cc:377] Logged in from keytab as oryx/127.4.40.3@KRBTEST.COM (short username oryx)
I20260811 06:56:55.209522 20703 webserver.cc:460] Webserver started at http://127.4.40.3:43423/ using document root <none> and password file <none>
I20260811 06:56:55.210191 20703 fs_manager.cc:359] Metadata directory not provided
I20260811 06:56:55.210309 20703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:56:55.215876 20703 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:56:55.218643 20721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:55.220005 20703 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:56:55.220268 20703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "f931796ff75f4c7aaffdd311bd273b63"
format_stamp: "Formatted at 2026-08-11 06:56:52 on dist-test-slave-7vjv"
server_key: "47f27e004ca4e36b189dd9adec727b25"
server_key_iv: "5bff39a9a71ed3dacc5ec129e7fc6d83"
server_key_version: "encryptionkey@0"
I20260811 06:56:55.221254 20703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:56:55.235344 20703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:56:55.240597 20703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:56:55.240931 20703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:56:55.242295 20703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:56:55.244230 20703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:56:55.244354 20703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:55.244472 20703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:56:55.244561 20703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:56:55.287432 20703 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:39863
I20260811 06:56:55.287528 20901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:39863 every 8 connection(s)
I20260811 06:56:55.289014 20703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:56:55.289565 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 20703
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:55.308012 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:55.290713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53024 (local address 127.4.40.62:36869)
0811 06:56:55.290919 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:55.290925 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:55.292490 (+ 1565us) server_negotiation.cc:407] Connection header received
0811 06:56:55.292697 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:55.292702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:55.292783 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:55.292894 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:55.295540 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.296651 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.298061 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.298476 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.298611 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.301330 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:56:55.301337 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:56:55.301350 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:56:55.301384 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:56:55.304612 (+ 3228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:55.305299 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:55.305307 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:55.305313 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:55.305488 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:56:55.305996 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:56:55.305999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:56:55.306001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:56:55.306503 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:56:55.306824 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 06:56:55.307232 (+ 408us) server_negotiation.cc:299] Negotiation successful
0811 06:56:55.307628 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 06:56:55.309532 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:55.292523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:35375 (local address 127.4.40.62:36869)
0811 06:56:55.297111 (+ 4588us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:55.297116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:55.297133 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:56:55.298440 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:55.298445 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:55.298538 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:55.298654 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:55.300614 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.301352 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.303357 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.303826 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.303976 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.309241 (+ 5265us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:35375: BlockingRecv error: recv got EOF from 127.4.40.3:35375 (error 108)
Metrics: {"server-negotiator.queue_time_us":4481,"thread_start_us":4374,"threads_started":1}
I20260811 06:56:55.310083 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:55.293147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:35375)
0811 06:56:55.296215 (+ 3068us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:55.296280 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:55.298182 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:55.298912 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:55.298924 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:55.299701 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:55.300422 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.300444 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.301510 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.301515 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.303186 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.303199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.304067 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.304076 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.304400 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.305019 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:55.305039 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:55.308985 (+ 3946us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":2673,"thread_start_us":2428,"threads_started":1}
W20260811 06:56:55.310943 20902 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:36869 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.40.62:36869: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:55.319551 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:55.311613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:55397)
0811 06:56:55.311864 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:55.311881 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:55.311985 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:55.312357 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:55.312361 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:55.312738 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:55.312953 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.312963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.313950 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.313954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.315291 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.315304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.315983 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.315988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.316194 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.316691 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:55.316707 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:55.319363 (+ 2656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151}
W20260811 06:56:55.319799 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:55.311764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:55397 (local address 127.4.40.62:36869)
0811 06:56:55.311917 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:55.311923 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:55.311953 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:56:55.312162 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:55.312166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:55.312218 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:55.312329 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:55.313089 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.313808 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.315427 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.315811 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.315958 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.319540 (+ 3582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:55397: BlockingRecv error: recv got EOF from 127.4.40.3:55397 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:55.328521 20910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:55.320320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:58749)
0811 06:56:55.320476 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:55.320490 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:55.320608 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:55.320961 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:55.320965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:55.321370 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:55.321603 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.321622 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.322600 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.322605 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.323491 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.323501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.324515 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.324521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.324693 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.325298 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:55.325340 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:55.328299 (+ 2959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:56:55.328698 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:55.320405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:58749 (local address 127.4.40.62:36869)
0811 06:56:55.320666 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:55.320672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:55.320690 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:56:55.320734 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:55.320738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:55.320812 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:55.320938 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:55.321753 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.322455 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.323632 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.324366 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.324501 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.328457 (+ 3956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:58749: BlockingRecv error: recv got EOF from 127.4.40.3:58749 (error 108)
Metrics: {"server-negotiator.queue_time_us":147}
W20260811 06:56:55.328886 20902 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:55.409832 20911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:55.399510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:47943)
0811 06:56:55.400188 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:55.400208 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:55.400411 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:55.400842 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:55.400848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:55.401506 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:55.401755 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.401767 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.402999 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.403006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.404245 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.404259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.405168 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.405175 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.405405 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.406144 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:55.406166 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:55.409446 (+ 3280us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":362,"threads_started":1}
W20260811 06:56:55.409946 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:55.399566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:47943 (local address 127.4.40.62:36869)
0811 06:56:55.399843 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:55.399851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:55.400343 (+ 492us) server_negotiation.cc:407] Connection header received
0811 06:56:55.400580 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:55.400584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:55.400673 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:55.400791 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:55.401934 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.402831 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.404442 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.404944 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.405115 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.409696 (+ 4581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:47943: BlockingRecv error: recv got EOF from 127.4.40.1:47943 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:55 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:55.868755 20913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:55.859182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:55859)
0811 06:56:55.859674 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:55.859690 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:55.859845 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:55.860228 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:55.860233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:55.860736 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:55.860945 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.860953 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.862000 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.862006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.863332 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:55.863351 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.864476 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.864481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:55.864638 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.865410 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:55.865430 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:55.868529 (+ 3099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":240,"threads_started":1}
W20260811 06:56:55.869010 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:55.859201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:55859 (local address 127.4.40.62:36869)
0811 06:56:55.859417 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:55.859426 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:55.859783 (+ 357us) server_negotiation.cc:407] Connection header received
0811 06:56:55.859979 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:55.859984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:55.860055 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:55.860202 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:55.861103 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.861850 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.863511 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:55.864284 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:55.864382 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:55.868740 (+ 4358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:55859: BlockingRecv error: recv got EOF from 127.4.40.2:55859 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 06:56:56 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:56 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:56.342932 20916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:56.329581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:35327)
0811 06:56:56.330141 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:56.330162 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:56.330275 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:56.330656 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:56.330661 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:56.331286 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:56.331492 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:56.331501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.332679 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.332687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:56.333640 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:56.333651 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.334424 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.334431 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:56.334634 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:56.338843 (+ 4209us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:56.338868 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:56.342704 (+ 3836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":299,"threads_started":1}
W20260811 06:56:56.343331 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:56.329642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:35327 (local address 127.4.40.62:36869)
0811 06:56:56.329842 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:56.329848 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:56.330259 (+ 411us) server_negotiation.cc:407] Connection header received
0811 06:56:56.330417 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:56.330423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:56.330503 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:56.330619 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:56.331658 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.332529 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.333795 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.334234 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.334394 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:56.342979 (+ 8585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:35327: BlockingRecv error: recv got EOF from 127.4.40.3:35327 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 06:56:56 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:56 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:56.419256 20917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:56.410834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:36623)
0811 06:56:56.411465 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:56.411481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:56.411645 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:56.412019 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:56.412024 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:56.412533 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:56.412728 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:56.412737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.413690 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.413696 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:56.414589 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:56.414600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.415323 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.415329 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:56.415501 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:56.416063 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:56.416080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:56.418868 (+ 2788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":345,"threads_started":1}
W20260811 06:56:56.419353 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:56.411069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:36623 (local address 127.4.40.62:36869)
0811 06:56:56.411333 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:56.411339 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:56.411568 (+ 229us) server_negotiation.cc:407] Connection header received
0811 06:56:56.411783 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:56.411788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:56.411858 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:56.411963 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:56.412874 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.413562 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.414733 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.415159 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.415290 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:56.419103 (+ 3813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:36623: BlockingRecv error: recv got EOF from 127.4.40.1:36623 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 06:56:56 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:56 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:56.878307 20918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:56.869640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:55705)
0811 06:56:56.870180 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:56.870196 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:56.870356 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:56.870709 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:56.870714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:56.871168 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:56.871374 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:56.871390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.872696 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.872701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:56.873676 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:56.873687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.874469 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.874475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:56.874644 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:56.875186 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:56.875203 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:56.877942 (+ 2739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":279,"threads_started":1}
W20260811 06:56:56.878371 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:56.869738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:55705 (local address 127.4.40.62:36869)
0811 06:56:56.869979 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:56.869986 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:56.870287 (+ 301us) server_negotiation.cc:407] Connection header received
0811 06:56:56.870486 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:56.870491 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:56.870572 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:56.870673 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:56.871531 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.872559 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.873821 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:56.874301 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:56.874436 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:56.878148 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:55705: BlockingRecv error: recv got EOF from 127.4.40.2:55705 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 06:56:57 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:57 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:57.353082 20919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:57.343817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:43483)
0811 06:56:57.344377 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:57.344397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:57.344579 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:57.345014 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:57.345020 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:57.345598 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:57.345863 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:57.345878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.347065 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.347069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:57.347968 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:57.347979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.348656 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.348662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:57.348815 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:57.349482 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:57.349505 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:57.352846 (+ 3341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":305,"threads_started":1}
W20260811 06:56:57.353302 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:57.344029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:43483 (local address 127.4.40.62:36869)
0811 06:56:57.344266 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:57.344274 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:57.344491 (+ 217us) server_negotiation.cc:407] Connection header received
0811 06:56:57.344723 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:57.344727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:57.344835 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:57.344944 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:57.346052 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.346933 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.348086 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.348512 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.348600 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:57.353033 (+ 4433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:43483: BlockingRecv error: recv got EOF from 127.4.40.3:43483 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 06:56:57 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:57 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:57.428751 20920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:57.420344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:53243)
0811 06:56:57.420879 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:57.420895 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:57.421005 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:57.421318 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:57.421321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:57.421713 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:57.421891 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:57.421898 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.422927 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.422931 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:57.423830 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:57.423841 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.424641 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.424645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:57.424765 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:57.425378 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:57.425403 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:57.428384 (+ 2981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":295,"threads_started":1}
W20260811 06:56:57.428879 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:57.420440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:53243 (local address 127.4.40.62:36869)
0811 06:56:57.420619 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:57.420624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:57.420963 (+ 339us) server_negotiation.cc:407] Connection header received
0811 06:56:57.421108 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:57.421113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:57.421182 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:57.421261 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:57.422019 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.422800 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.423950 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.424498 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.424608 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:57.428643 (+ 4035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:53243: BlockingRecv error: recv got EOF from 127.4.40.1:53243 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:56:57 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:57 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:57.887768 20921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:57.879449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:34807)
0811 06:56:57.880018 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:57.880039 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:57.880218 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:57.880560 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:57.880564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:57.880953 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:57.881116 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:57.881125 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.882163 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.882170 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:57.883175 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:57.883187 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.883897 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.883902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:57.884109 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:57.884737 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:57.884754 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:57.887532 (+ 2778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":308,"threads_started":1}
W20260811 06:56:57.887939 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:57.879531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:34807 (local address 127.4.40.62:36869)
0811 06:56:57.879769 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:57.879777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:57.880125 (+ 348us) server_negotiation.cc:407] Connection header received
0811 06:56:57.880340 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:57.880346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:57.880413 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:57.880526 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:57.881263 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.882026 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.883324 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:57.883740 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:57.883873 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:57.887691 (+ 3818us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:34807: BlockingRecv error: recv got EOF from 127.4.40.2:34807 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:56:58 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:58 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:58.362824 20922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:58.353968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:55685)
0811 06:56:58.354499 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:58.354541 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:58.354672 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:58.354978 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:58.354981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:58.355327 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:58.355501 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:58.355509 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.357010 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.357018 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:58.358098 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:58.358111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.358778 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.358782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:58.358899 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:58.359448 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:58.359464 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:58.362468 (+ 3004us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":298,"threads_started":1}
W20260811 06:56:58.362927 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:58.354082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:55685 (local address 127.4.40.62:36869)
0811 06:56:58.354276 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:58.354280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:58.354631 (+ 351us) server_negotiation.cc:407] Connection header received
0811 06:56:58.354766 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:58.354769 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:58.354843 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:58.354926 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:58.355623 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.356838 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.358235 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.358657 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.358752 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:58.362682 (+ 3930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:55685: BlockingRecv error: recv got EOF from 127.4.40.3:55685 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 06:56:58 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:58 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:58.438382 20923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:58.429865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:36927)
0811 06:56:58.430424 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:58.430441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:58.430562 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:58.430846 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:58.430849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:58.431233 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:58.431385 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:58.431392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.432420 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.432425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:58.433571 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:58.433584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.434252 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.434255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:58.434387 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:58.435002 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:58.435019 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:58.438033 (+ 3014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":317,"threads_started":1}
W20260811 06:56:58.438479 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:58.429918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:36927 (local address 127.4.40.62:36869)
0811 06:56:58.430097 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:58.430102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:58.430522 (+ 420us) server_negotiation.cc:407] Connection header received
0811 06:56:58.430655 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:58.430659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:58.430718 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:58.430798 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:58.431521 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.432292 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.433697 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.434130 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.434221 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:58.438224 (+ 4003us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:36927: BlockingRecv error: recv got EOF from 127.4.40.1:36927 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 06:56:58 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:58 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:58.897171 20924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:58.888782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:60477)
0811 06:56:58.889327 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:58.889343 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:58.889456 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:58.889752 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:58.889755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:58.890135 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:58.890288 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:58.890296 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.891280 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.891284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:58.892380 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:58.892394 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.893177 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.893183 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:58.893380 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:58.893951 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:58.893968 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:58.896936 (+ 2968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":302,"threads_started":1}
W20260811 06:56:58.897336 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:58.888837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:60477 (local address 127.4.40.62:36869)
0811 06:56:58.889023 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:58.889028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:58.889416 (+ 388us) server_negotiation.cc:407] Connection header received
0811 06:56:58.889559 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:58.889562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:58.889621 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:58.889702 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:58.890409 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.891154 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.892528 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:58.893011 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:58.893157 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:58.897125 (+ 3968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:60477: BlockingRecv error: recv got EOF from 127.4.40.2:60477 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:56:59 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:59 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:59.372279 20925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:59.363838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:59941)
0811 06:56:59.364406 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:59.364426 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:59.364545 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:59.364856 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:59.364859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:59.365239 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:59.365392 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:59.365400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.366355 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.366358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:59.367323 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:59.367334 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.367992 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.367996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:59.368226 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:59.368897 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:59.368914 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:59.371907 (+ 2993us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":320,"threads_started":1}
W20260811 06:56:59.372401 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:59.363917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:59941 (local address 127.4.40.62:36869)
0811 06:56:59.364146 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:59.364153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:59.364507 (+ 354us) server_negotiation.cc:407] Connection header received
0811 06:56:59.364648 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:59.364651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:59.364725 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:59.364805 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:59.365515 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.366236 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.367451 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.367863 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.367952 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:59.372140 (+ 4188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:59941: BlockingRecv error: recv got EOF from 127.4.40.3:59941 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 06:56:59 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:59 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:59.447992 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:59.439365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:44669)
0811 06:56:59.439929 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:59.439945 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:59.440058 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:59.440388 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:59.440391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:59.440853 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:59.441010 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:59.441018 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.441976 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.441980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:59.442913 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:59.442923 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.443941 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.443948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:59.444111 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:59.444897 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:59.444924 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:59.447748 (+ 2824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":296,"threads_started":1}
W20260811 06:56:59.448182 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:59.439459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:44669 (local address 127.4.40.62:36869)
0811 06:56:59.439666 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:59.439670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:59.440024 (+ 354us) server_negotiation.cc:407] Connection header received
0811 06:56:59.440195 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:59.440199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:59.440258 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:59.440339 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:59.441128 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.441847 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.443058 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.443746 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.443903 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:59.447953 (+ 4050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:44669: BlockingRecv error: recv got EOF from 127.4.40.1:44669 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 06:56:59 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:56:59 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:56:59.907020 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:56:59.898251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:41711)
0811 06:56:59.898837 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 06:56:59.898853 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:56:59.898963 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:56:59.899277 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:56:59.899280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:56:59.899733 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:56:59.899904 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:59.899912 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.900975 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.900978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:59.901891 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:56:59.901901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.902617 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.902620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:56:59.902734 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:59.903329 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:56:59.903345 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:56:59.906671 (+ 3326us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":303,"threads_started":1}
W20260811 06:56:59.907150 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:56:59.898414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:41711 (local address 127.4.40.62:36869)
0811 06:56:59.898598 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:56:59.898602 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:56:59.898926 (+ 324us) server_negotiation.cc:407] Connection header received
0811 06:56:59.899059 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:56:59.899062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:56:59.899121 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:56:59.899198 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:56:59.900029 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.900847 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.902053 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:56:59.902478 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:56:59.902592 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:56:59.906910 (+ 4318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:41711: BlockingRecv error: recv got EOF from 127.4.40.2:41711 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 06:57:00 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:00 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:00.381177 20929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:00.373165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:51541)
0811 06:57:00.373701 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:00.373721 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:00.373828 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:00.374119 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:00.374123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:00.374544 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:00.374704 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:00.374712 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.375722 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.375726 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:00.376705 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:00.376715 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.377332 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.377336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:00.377451 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:00.378024 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:00.378041 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:00.380935 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":272,"threads_started":1}
W20260811 06:57:00.381376 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:00.373262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:51541 (local address 127.4.40.62:36869)
0811 06:57:00.373426 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:00.373431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:00.373790 (+ 359us) server_negotiation.cc:407] Connection header received
0811 06:57:00.373918 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:00.373922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:00.373991 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:00.374071 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:00.374820 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.375595 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.376819 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.377213 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.377303 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:00.381124 (+ 3821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:51541: BlockingRecv error: recv got EOF from 127.4.40.3:51541 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:57:00 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:00 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:00.458272 20931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:00.449015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:59917)
0811 06:57:00.449573 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:00.449589 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:00.449747 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:00.450118 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:00.450125 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:00.450554 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:00.450706 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:00.450715 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.451761 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.451767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:00.453119 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:00.453141 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.454208 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.454216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:00.454410 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:00.455010 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:00.455029 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:00.458054 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":333,"threads_started":1}
W20260811 06:57:00.458460 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:00.449090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:59917 (local address 127.4.40.62:36869)
0811 06:57:00.449370 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:00.449378 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:00.449677 (+ 299us) server_negotiation.cc:407] Connection header received
0811 06:57:00.449899 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:00.449908 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:00.449968 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:00.450075 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:00.450874 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.451599 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.453302 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.454022 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.454168 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:00.458232 (+ 4064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:59917: BlockingRecv error: recv got EOF from 127.4.40.1:59917 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 11 06:57:00 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:00 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:00.916049 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:00.907976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:55831)
0811 06:57:00.908553 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:00.908572 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:00.908691 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:00.908987 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:00.908990 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:00.909402 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:00.909551 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:00.909572 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.910531 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.910534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:00.911432 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:00.911441 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.912225 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.912230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:00.912353 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:00.912902 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:00.912919 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:00.915777 (+ 2858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":344,"threads_started":1}
W20260811 06:57:00.916268 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:00.908064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:55831 (local address 127.4.40.62:36869)
0811 06:57:00.908253 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:00.908257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:00.908650 (+ 393us) server_negotiation.cc:407] Connection header received
0811 06:57:00.908784 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:00.908788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:00.908847 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:00.908926 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:00.909681 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.910414 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.911562 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:00.912019 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:00.912133 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:00.915994 (+ 3861us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:55831: BlockingRecv error: recv got EOF from 127.4.40.2:55831 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 06:57:01 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:01 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:01.394181 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:01.382232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:53093)
0811 06:57:01.382807 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:01.382825 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:01.382982 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:01.383319 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:01.383325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:01.383771 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:01.383923 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:01.383931 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.385252 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.385258 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:01.386270 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:01.386281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.386981 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.386987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:01.387163 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:01.390315 (+ 3152us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:01.390333 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:01.393838 (+ 3505us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":362,"threads_started":1}
W20260811 06:57:01.394310 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:01.382357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:53093 (local address 127.4.40.62:36869)
0811 06:57:01.382555 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:01.382564 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:01.382911 (+ 347us) server_negotiation.cc:407] Connection header received
0811 06:57:01.383105 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:01.383110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:01.383179 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:01.383279 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:01.384078 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.385103 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.386420 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.386823 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.386958 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:01.394031 (+ 7073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:53093: BlockingRecv error: recv got EOF from 127.4.40.3:53093 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:57:01 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:01 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:01.467845 20935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:01.459483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:38141)
0811 06:57:01.460230 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:01.460250 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:01.460416 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:01.460767 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:01.460772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:01.461262 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:01.461434 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:01.461441 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.462447 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.462453 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:01.463330 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:01.463340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.464032 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.464037 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:01.464256 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:01.464770 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:01.464787 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:01.467602 (+ 2815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":270,"threads_started":1}
W20260811 06:57:01.468003 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:01.459539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:38141 (local address 127.4.40.62:36869)
0811 06:57:01.459750 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:01.459758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:01.460345 (+ 587us) server_negotiation.cc:407] Connection header received
0811 06:57:01.460555 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:01.460561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:01.460633 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:01.460735 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:01.461584 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.462311 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.463477 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.463881 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.464008 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:01.467786 (+ 3778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:38141: BlockingRecv error: recv got EOF from 127.4.40.1:38141 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:57:01 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:01 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:01.926133 20937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:01.917235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:47449)
0811 06:57:01.917891 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:01.917908 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:01.918037 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:01.918338 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:01.918341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:01.918771 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:01.918946 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:01.918954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.920000 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.920019 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:01.921464 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:01.921500 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.922296 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.922300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:01.922407 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:01.922961 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:01.922978 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:01.925874 (+ 2896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":300,"threads_started":1}
W20260811 06:57:01.926347 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:01.917298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:47449 (local address 127.4.40.62:36869)
0811 06:57:01.917530 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:01.917535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:01.917994 (+ 459us) server_negotiation.cc:407] Connection header received
0811 06:57:01.918139 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:01.918142 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:01.918203 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:01.918290 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:01.919072 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.919839 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.921647 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:01.922155 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:01.922252 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:01.926076 (+ 3824us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:47449: BlockingRecv error: recv got EOF from 127.4.40.2:47449 (error 108)
Metrics: {"server-negotiator.queue_time_us":116}
Aug 11 06:57:02 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:02 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:02.403852 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:02.395412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:40311)
0811 06:57:02.396161 (+ 749us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:02.396187 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:02.396338 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:02.396732 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:02.396737 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:02.397293 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:02.397471 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:02.397480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.398455 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.398458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:02.399361 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:02.399371 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.400164 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.400171 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:02.400302 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:02.400848 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:02.400865 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:02.403610 (+ 2745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":286,"threads_started":1}
W20260811 06:57:02.403993 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:02.395649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:40311 (local address 127.4.40.62:36869)
0811 06:57:02.395881 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:02.395887 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:02.396283 (+ 396us) server_negotiation.cc:407] Connection header received
0811 06:57:02.396491 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:02.396496 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:02.396578 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:02.396669 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:02.397609 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.398337 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.399482 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.399937 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.400059 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:02.403783 (+ 3724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:40311: BlockingRecv error: recv got EOF from 127.4.40.3:40311 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 06:57:02 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:02 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:02.477172 20939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:02.468745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:54443)
0811 06:57:02.469276 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:02.469292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:02.469440 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:02.469772 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:02.469777 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:02.470248 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:02.470403 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:02.470412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.471513 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.471520 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:02.472547 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:02.472559 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.473330 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.473336 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:02.473498 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:02.474063 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:02.474080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:02.476842 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":287,"threads_started":1}
W20260811 06:57:02.477280 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:02.468933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:54443 (local address 127.4.40.62:36869)
0811 06:57:02.469180 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:02.469187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:02.469374 (+ 187us) server_negotiation.cc:407] Connection header received
0811 06:57:02.469571 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:02.469581 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:02.469640 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:02.469739 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:02.470560 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.471339 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.472690 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.473143 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.473277 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:02.477049 (+ 3772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:54443: BlockingRecv error: recv got EOF from 127.4.40.1:54443 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 06:57:02 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:02 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:02.936245 20940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:02.927089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:40015)
0811 06:57:02.927634 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:02.927649 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:02.927821 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:02.928228 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:02.928232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:02.928678 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:02.928833 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:02.928842 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.930038 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.930045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:02.930968 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:02.930979 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.931899 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.931906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:02.932139 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:02.932987 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:02.933009 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:02.935879 (+ 2870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":284,"threads_started":1}
W20260811 06:57:02.936432 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:02.927170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:40015 (local address 127.4.40.62:36869)
0811 06:57:02.927414 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:02.927422 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:02.927739 (+ 317us) server_negotiation.cc:407] Connection header received
0811 06:57:02.927982 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:02.927986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:02.928063 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:02.928190 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:02.928997 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.929886 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.931119 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:02.931685 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:02.931838 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:02.936158 (+ 4320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:40015: BlockingRecv error: recv got EOF from 127.4.40.2:40015 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 06:57:03 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:03 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:03.413450 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:03.404825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:47551)
0811 06:57:03.405390 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:03.405424 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:03.405539 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:03.405832 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:03.405835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:03.406272 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:03.406430 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:03.406437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.407389 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.407393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:03.408337 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:03.408348 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.409115 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.409122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:03.409299 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:03.410069 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:03.410093 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:03.413090 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":316,"threads_started":1}
W20260811 06:57:03.413559 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:03.404906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:47551 (local address 127.4.40.62:36869)
0811 06:57:03.405083 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:03.405088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:03.405494 (+ 406us) server_negotiation.cc:407] Connection header received
0811 06:57:03.405634 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:03.405637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:03.405698 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:03.405779 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:03.406559 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.407266 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.408458 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.408868 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.409041 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:03.413312 (+ 4271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:47551: BlockingRecv error: recv got EOF from 127.4.40.3:47551 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 06:57:03 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:03 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:03.487267 20943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:03.478245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:40077)
0811 06:57:03.478819 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:03.478835 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:03.478961 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:03.479255 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:03.479258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:03.479690 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:03.479866 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:03.479875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.481319 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.481328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:03.482445 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:03.482457 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.483118 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.483122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:03.483246 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:03.483775 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:03.483792 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:03.486881 (+ 3089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":325,"threads_started":1}
W20260811 06:57:03.487361 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:03.478300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:40077 (local address 127.4.40.62:36869)
0811 06:57:03.478475 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:03.478481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:03.478908 (+ 427us) server_negotiation.cc:407] Connection header received
0811 06:57:03.479061 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:03.479064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:03.479124 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:03.479206 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:03.480029 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.481162 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.482576 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.482973 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.483077 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:03.487115 (+ 4038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:40077: BlockingRecv error: recv got EOF from 127.4.40.1:40077 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 06:57:03 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:03 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:03.945705 20944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:03.937251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:60727)
0811 06:57:03.937845 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:03.937862 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:03.938001 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:03.938287 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:03.938290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:03.938713 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:03.938891 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:03.938899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.939899 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.939903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:03.940861 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:03.940872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.941555 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.941560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:03.941678 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:03.942245 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:03.942270 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:03.945448 (+ 3178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":334,"threads_started":1}
W20260811 06:57:03.945868 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:03.937320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:60727 (local address 127.4.40.62:36869)
0811 06:57:03.937480 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:03.937485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:03.937940 (+ 455us) server_negotiation.cc:407] Connection header received
0811 06:57:03.938100 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:03.938103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:03.938159 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:03.938238 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:03.939011 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.939776 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.940978 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:03.941418 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:03.941519 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:03.945641 (+ 4122us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:60727: BlockingRecv error: recv got EOF from 127.4.40.2:60727 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:57:04 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:04 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:04.423729 20945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:04.414447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:37493)
0811 06:57:04.414987 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:04.415008 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:04.415177 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:04.415521 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:04.415527 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:04.415986 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:04.416262 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:04.416272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.417257 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.417265 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:04.418719 (+ 1454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:04.418733 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.419481 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.419488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:04.419694 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:04.420542 (+ 848us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:04.420560 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:04.423475 (+ 2915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":291,"threads_started":1}
W20260811 06:57:04.423888 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:04.414521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:37493 (local address 127.4.40.62:36869)
0811 06:57:04.414770 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:04.414779 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:04.415103 (+ 324us) server_negotiation.cc:407] Connection header received
0811 06:57:04.415304 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:04.415312 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:04.415372 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:04.415478 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:04.416403 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.417116 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.418905 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.419317 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.419450 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:04.423690 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:37493: BlockingRecv error: recv got EOF from 127.4.40.3:37493 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 06:57:04 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:04 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:04.496493 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:04.488429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:58065)
0811 06:57:04.489192 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:04.489209 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:04.489318 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:04.489575 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:04.489578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:04.489954 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:04.490103 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:04.490111 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.491074 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.491077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:04.491982 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:04.491992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.492809 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.492813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:04.492940 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:04.493466 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:04.493481 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:04.496276 (+ 2795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":528,"threads_started":1}
W20260811 06:57:04.496740 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:04.488595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:58065 (local address 127.4.40.62:36869)
0811 06:57:04.488905 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:04.488914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:04.489288 (+ 374us) server_negotiation.cc:407] Connection header received
0811 06:57:04.489414 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:04.489417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:04.489488 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:04.489558 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:04.490234 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.490948 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.492148 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.492673 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.492765 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:04.496451 (+ 3686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:58065: BlockingRecv error: recv got EOF from 127.4.40.1:58065 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 06:57:04 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:04 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:04.954952 20947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:04.946905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:45261)
0811 06:57:04.947458 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:04.947475 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:04.947585 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:04.947867 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:04.947870 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:04.948326 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:04.948482 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:04.948490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.949456 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.949460 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:04.950395 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:04.950405 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.951025 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.951029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:04.951149 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:04.951729 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:04.951746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:04.954734 (+ 2988us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":322,"threads_started":1}
W20260811 06:57:04.955127 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:04.946993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:45261 (local address 127.4.40.62:36869)
0811 06:57:04.947179 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:04.947184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:04.947548 (+ 364us) server_negotiation.cc:407] Connection header received
0811 06:57:04.947677 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:04.947681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:04.947737 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:04.947818 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:04.948611 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.949327 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.950512 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:04.950892 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:04.950988 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:04.954901 (+ 3913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:45261: BlockingRecv error: recv got EOF from 127.4.40.2:45261 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 06:57:05 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:05 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:05.433588 20948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:05.424622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:39489)
0811 06:57:05.425236 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:05.425262 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:05.425405 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:05.425759 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:05.425764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:05.426173 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:05.426325 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:05.426333 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.427447 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.427452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:05.428477 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:05.428489 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.429280 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.429286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:05.429467 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:05.430126 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:05.430143 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:05.433354 (+ 3211us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":341,"threads_started":1}
W20260811 06:57:05.433796 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:05.424788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:39489 (local address 127.4.40.62:36869)
0811 06:57:05.425015 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:05.425023 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:05.425342 (+ 319us) server_negotiation.cc:407] Connection header received
0811 06:57:05.425554 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:05.425559 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:05.425628 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:05.425727 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:05.426484 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.427305 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.428618 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.429134 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.429274 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:05.433554 (+ 4280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:39489: BlockingRecv error: recv got EOF from 127.4.40.3:39489 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 11 06:57:05 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:05 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:05.507076 20950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:05.497541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:50509)
0811 06:57:05.498157 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:05.498177 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:05.498312 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:05.498650 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:05.498654 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:05.499090 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:05.499264 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:05.499271 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.500353 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.500357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:05.501983 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:05.502011 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.502859 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.502865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:05.503070 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:05.503830 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:05.503847 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:05.506709 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":359,"threads_started":1}
W20260811 06:57:05.507164 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:05.497787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:50509 (local address 127.4.40.62:36869)
0811 06:57:05.498055 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:05.498063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:05.498261 (+ 198us) server_negotiation.cc:407] Connection header received
0811 06:57:05.498420 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:05.498424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:05.498485 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:05.498563 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:05.499388 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.500239 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.502142 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.502700 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.502838 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:05.506926 (+ 4088us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:50509: BlockingRecv error: recv got EOF from 127.4.40.1:50509 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
Aug 11 06:57:05 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:05 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:05.963851 20952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:05.955899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:56971)
0811 06:57:05.956424 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:05.956441 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:05.956553 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:05.956805 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:05.956808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:05.957224 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:05.957401 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:05.957409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.958364 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.958368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:05.959224 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:05.959234 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.959958 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.959975 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:05.960145 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:05.960762 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:05.960779 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:05.963586 (+ 2807us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":302,"threads_started":1}
W20260811 06:57:05.964073 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:05.955974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:56971 (local address 127.4.40.62:36869)
0811 06:57:05.956234 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:05.956241 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:05.956529 (+ 288us) server_negotiation.cc:407] Connection header received
0811 06:57:05.956651 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:05.956655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:05.956714 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:05.956792 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:05.957525 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.958249 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.959370 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:05.959824 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:05.959914 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:05.963793 (+ 3879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:56971: BlockingRecv error: recv got EOF from 127.4.40.2:56971 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:57:06 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:06 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:06.447777 20953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:06.434541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:58139)
0811 06:57:06.435063 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:06.435083 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:06.435264 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:06.435624 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:06.435633 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:06.436070 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:06.436335 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:06.436345 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.437431 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.437437 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:06.438306 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:06.438318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.439049 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.439056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:06.439234 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:06.444222 (+ 4988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:06.444240 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:06.447399 (+ 3159us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":284,"threads_started":1}
W20260811 06:57:06.447826 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:06.434639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:58139 (local address 127.4.40.62:36869)
0811 06:57:06.434878 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:06.434885 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:06.435188 (+ 303us) server_negotiation.cc:407] Connection header received
0811 06:57:06.435389 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:06.435394 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:06.435467 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:06.435584 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:06.436506 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.437281 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.438461 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.438875 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.439025 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:06.447596 (+ 8571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:58139: BlockingRecv error: recv got EOF from 127.4.40.3:58139 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 06:57:06 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:06 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:06.517524 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:06.508298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:40167)
0811 06:57:06.508878 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:06.508894 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:06.509022 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:06.509337 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:06.509340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:06.509751 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:06.509910 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:06.509918 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.510996 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.511000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:06.511928 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:06.511940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.512700 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.512705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:06.512837 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:06.513554 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:06.513573 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:06.517290 (+ 3717us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":314,"threads_started":1}
W20260811 06:57:06.517745 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:06.508378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:40167 (local address 127.4.40.62:36869)
0811 06:57:06.508587 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:06.508591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:06.508986 (+ 395us) server_negotiation.cc:407] Connection header received
0811 06:57:06.509116 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:06.509120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:06.509177 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:06.509275 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:06.510063 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.510859 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.512058 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.512572 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.512666 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:06.517477 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:40167: BlockingRecv error: recv got EOF from 127.4.40.1:40167 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 06:57:06 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:06 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:06.973418 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:06.964915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:38279)
0811 06:57:06.965487 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:06.965504 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:06.965625 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:06.965972 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:06.965979 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:06.966388 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:06.966552 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:06.966560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.967836 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.967841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:06.968829 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:06.968841 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.969509 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.969514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:06.969640 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:06.970309 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:06.970327 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:06.973195 (+ 2868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":291,"threads_started":1}
W20260811 06:57:06.973613 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:06.964967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:38279 (local address 127.4.40.62:36869)
0811 06:57:06.965205 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:06.965215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:06.965591 (+ 376us) server_negotiation.cc:407] Connection header received
0811 06:57:06.965740 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:06.965744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:06.965807 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:06.965887 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:06.966696 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.967705 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.968975 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:06.969376 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:06.969469 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:06.973387 (+ 3918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:38279: BlockingRecv error: recv got EOF from 127.4.40.2:38279 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 06:57:07 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:07 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:07.457554 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:07.448797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:34003)
0811 06:57:07.449391 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:07.449412 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:07.449538 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:07.449832 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:07.449835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:07.450216 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:07.450377 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:07.450385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.451597 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.451601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:07.452589 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:07.452602 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.453249 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.453253 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:07.453373 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:07.454130 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:07.454148 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:07.457326 (+ 3178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":341,"threads_started":1}
W20260811 06:57:07.457796 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:07.448851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:34003 (local address 127.4.40.62:36869)
0811 06:57:07.449009 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:07.449015 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:07.449482 (+ 467us) server_negotiation.cc:407] Connection header received
0811 06:57:07.449633 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:07.449636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:07.449695 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:07.449772 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:07.450509 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.451451 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.452717 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.453126 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.453218 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:07.457536 (+ 4318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:34003: BlockingRecv error: recv got EOF from 127.4.40.3:34003 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 06:57:07 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:07 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:07.527516 20959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:07.518505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:42137)
0811 06:57:07.519165 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:07.519182 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:07.519347 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:07.519719 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:07.519723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:07.520246 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:07.520406 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:07.520415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.521489 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.521497 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:07.522414 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:07.522425 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.523150 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.523157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:07.523318 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:07.523886 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:07.523903 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:07.527265 (+ 3362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":376,"threads_started":1}
W20260811 06:57:07.527925 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:07.518601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:42137 (local address 127.4.40.62:36869)
0811 06:57:07.518954 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:07.518961 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:07.519276 (+ 315us) server_negotiation.cc:407] Connection header received
0811 06:57:07.519486 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:07.519495 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:07.519555 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:07.519683 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:07.520572 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.521337 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.522568 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.522969 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.523111 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:07.527693 (+ 4582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:42137: BlockingRecv error: recv got EOF from 127.4.40.1:42137 (error 108)
Metrics: {"server-negotiator.queue_time_us":177}
Aug 11 06:57:07 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:07 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:07.983306 20960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:07.974590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:44815)
0811 06:57:07.975205 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:07.975222 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:07.975384 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:07.975754 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:07.975761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:07.976382 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:07.976545 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:07.976554 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.977729 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.977733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:07.978662 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:07.978672 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.979418 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.979422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:07.979539 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:07.980130 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:07.980147 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:07.983077 (+ 2930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":337,"threads_started":1}
W20260811 06:57:07.983493 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:07.974784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:44815 (local address 127.4.40.62:36869)
0811 06:57:07.974985 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:07.974990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:07.975310 (+ 320us) server_negotiation.cc:407] Connection header received
0811 06:57:07.975542 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:07.975545 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:07.975606 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:07.975704 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:07.976718 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.977594 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.978787 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:07.979239 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:07.979347 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:07.983256 (+ 3909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:44815: BlockingRecv error: recv got EOF from 127.4.40.2:44815 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 06:57:08 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:08 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:08.468772 20961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:08.458627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:38437)
0811 06:57:08.459355 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:08.459380 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:08.459519 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:08.459852 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:08.459856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:08.460387 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:08.460670 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:08.460689 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.462223 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.462230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:08.463429 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:08.463450 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.464455 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.464461 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:08.464667 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:08.465269 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:08.465290 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:08.468502 (+ 3212us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":397,"threads_started":1}
W20260811 06:57:08.468839 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:08.458692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38437 (local address 127.4.40.62:36869)
0811 06:57:08.458876 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:08.458881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:08.459463 (+ 582us) server_negotiation.cc:407] Connection header received
0811 06:57:08.459642 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:08.459645 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:08.459703 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:08.459787 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:08.460934 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.462052 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.463608 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.464052 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.464220 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:08.468672 (+ 4452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:38437: BlockingRecv error: recv got EOF from 127.4.40.3:38437 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 06:57:08 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:08 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:08.537984 20962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:08.528618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:33009)
0811 06:57:08.529192 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:08.529208 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:08.529381 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:08.529734 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:08.529739 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:08.530181 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:08.530353 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:08.530362 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.531417 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.531423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:08.532691 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:08.532708 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.533809 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.533816 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:08.533954 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:08.534663 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:08.534681 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:08.537626 (+ 2945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":276,"threads_started":1}
W20260811 06:57:08.538066 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:08.528684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:33009 (local address 127.4.40.62:36869)
0811 06:57:08.529019 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:08.529025 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:08.529309 (+ 284us) server_negotiation.cc:407] Connection header received
0811 06:57:08.529516 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:08.529521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:08.529595 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:08.529698 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:08.530517 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.531259 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.532875 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.533637 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.533776 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:08.537825 (+ 4049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:33009: BlockingRecv error: recv got EOF from 127.4.40.1:33009 (error 108)
Metrics: {"server-negotiator.queue_time_us":146}
Aug 11 06:57:08 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:08 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:08.992560 20964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:08.984302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:43453)
0811 06:57:08.984862 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:08.984878 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:08.985080 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:08.985457 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:08.985476 (+ 19us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:08.985890 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:08.986066 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:08.986075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.987214 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.987218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:08.988228 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:08.988240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.988890 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.988906 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:08.989038 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:08.989583 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:08.989599 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:08.992294 (+ 2695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":317,"threads_started":1}
W20260811 06:57:08.992640 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:08.984326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:43453 (local address 127.4.40.62:36869)
0811 06:57:08.984534 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:08.984543 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:08.984998 (+ 455us) server_negotiation.cc:407] Connection header received
0811 06:57:08.985196 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:08.985200 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:08.985294 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:08.985398 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:08.986285 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.987074 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.988363 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:08.988762 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:08.988864 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:08.992471 (+ 3607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:43453: BlockingRecv error: recv got EOF from 127.4.40.2:43453 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 06:57:09 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:09 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.3@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:09.478682 20965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:09.469708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.3:57877)
0811 06:57:09.470196 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:09.470218 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:09.470331 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:09.470686 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:09.470689 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:09.471270 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:09.471545 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:09.471556 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.472873 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.472878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:09.473798 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:09.473809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.474522 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.474526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:09.474647 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:09.475336 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:09.475353 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:09.478422 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":274,"threads_started":1}
W20260811 06:57:09.478888 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:09.469846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:57877 (local address 127.4.40.62:36869)
0811 06:57:09.470089 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:09.470095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:09.470301 (+ 206us) server_negotiation.cc:407] Connection header received
0811 06:57:09.470433 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:09.470437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:09.470537 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:09.470646 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:09.471701 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.472732 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.473921 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.474394 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.474496 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:09.478631 (+ 4135us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.3:57877: BlockingRecv error: recv got EOF from 127.4.40.3:57877 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 06:57:09 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:09 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.1@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:09.547730 20967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:09.539204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.1:55265)
0811 06:57:09.539789 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:09.539805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:09.539916 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:09.540232 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:09.540235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:09.540662 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:09.540860 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:09.540868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.542204 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.542208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:09.543207 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:09.543219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.543890 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.543893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:09.544014 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:09.544694 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:09.544710 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:09.547494 (+ 2784us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":355,"threads_started":1}
W20260811 06:57:09.547827 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:09.539371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:55265 (local address 127.4.40.62:36869)
0811 06:57:09.539600 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:09.539605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:09.539882 (+ 277us) server_negotiation.cc:407] Connection header received
0811 06:57:09.540019 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:09.540023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:09.540083 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:09.540187 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:09.541020 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.542041 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.543335 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.543763 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.543854 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:09.547659 (+ 3805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.1:55265: BlockingRecv error: recv got EOF from 127.4.40.1:55265 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 06:57:09 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 06:57:09 dist-test-slave-7vjv krb5kdc[19208](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.40.2@KRBTEST.COM for oryx/127.4.40.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 06:57:10.002218 20969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:09.993767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36869 (local address 127.4.40.2:46161)
0811 06:57:09.994313 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:09.994329 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:09.994488 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:09.994883 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:09.994888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:09.995360 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:09.995570 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:09.995580 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.996731 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.996737 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:09.997628 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:09.997639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.998373 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.998379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:09.998552 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:09.999129 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:09.999146 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:10.001950 (+ 2804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.40.62:36869: Server oryx/127.4.40.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":306,"threads_started":1}
W20260811 06:57:10.002457 20909 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:09.993863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:46161 (local address 127.4.40.62:36869)
0811 06:57:09.994057 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:09.994064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:09.994420 (+ 356us) server_negotiation.cc:407] Connection header received
0811 06:57:09.994645 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:09.994649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:09.994713 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:09.994841 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:09.995725 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.996571 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.997778 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:09.998215 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:09.998349 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:10.002178 (+ 3829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.40.2:46161: BlockingRecv error: recv got EOF from 127.4.40.2:46161 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 06:57:10.291122 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20214
I20260811 06:57:10.304000 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:10.292237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47844 (local address 127.4.40.1:35375)
0811 06:57:10.292817 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:10.292824 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:10.292850 (+ 26us) server_negotiation.cc:407] Connection header received
0811 06:57:10.292915 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:10.292921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:10.293139 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:10.293298 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:10.295362 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:10.296547 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:10.297681 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:10.298196 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:10.298289 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:10.300840 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:10.300848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:10.300861 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:10.300894 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:10.303339 (+ 2445us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.40.1@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:57:10.303723 (+ 384us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47844: Request ticket server kudu/127.4.40.1@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":299,"threads_started":1}
W20260811 06:57:10.304136 20976 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47844: Request ticket server kudu/127.4.40.1@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:57:10.304136 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20214: Not authorized: Client connection negotiation failed: client connection to 127.4.40.1:35375: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.40.1@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:57:10.304374 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 20214
I20260811 06:57:10.331971 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20476
I20260811 06:57:10.347771 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:10.333101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32822 (local address 127.4.40.2:40255)
0811 06:57:10.333763 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:10.333770 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:10.333801 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:57:10.333865 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:10.333871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:10.334103 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:10.334253 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:10.335839 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:10.338110 (+ 2271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:10.339849 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:10.340870 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:10.340977 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:10.344420 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:10.344429 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:10.344443 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:10.344487 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:10.347201 (+ 2714us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.40.2@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:57:10.347468 (+ 267us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:32822: Request ticket server kudu/127.4.40.2@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":332,"threads_started":1}
W20260811 06:57:10.347856 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20476: Not authorized: Client connection negotiation failed: client connection to 127.4.40.2:40255: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.40.2@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:57:10.347995 20977 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:32822: Request ticket server kudu/127.4.40.2@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:57:10.348030 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 20476
I20260811 06:57:10.383371 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20703
I20260811 06:57:10.395640 20982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:10.384544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.4.40.3:39863)
0811 06:57:10.385161 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:10.385169 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:10.385196 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:57:10.385260 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:10.385266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:10.385531 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:10.385710 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:10.387212 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:10.388377 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:10.389617 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:10.390147 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:10.390243 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:10.392713 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:10.392721 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:10.392733 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:10.392774 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:10.395116 (+ 2342us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.40.3@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 06:57:10.395356 (+ 240us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41040: Request ticket server kudu/127.4.40.3@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":446,"thread_start_us":336,"threads_started":1}
W20260811 06:57:10.395746 20982 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41040: Request ticket server kudu/127.4.40.3@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 06:57:10.395782 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20703: Not authorized: Client connection negotiation failed: client connection to 127.4.40.3:39863: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.40.3@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 06:57:10.395970 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 20703
I20260811 06:57:10.424432 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20102
I20260811 06:57:10.425731 20206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:10.546756 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 20102
2026-08-11T06:57:10Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21684 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T06:57:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:57:10Z Disabled control of system clock
I20260811 06:57:10.650157 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36241
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 06:57:10.991829 20992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:10.992169 20992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:10.992282 20992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:10.999229 20992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:57:10.999334 20992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:10.999373 20992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:10.999423 20992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:57:10.999457 20992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:57:11.009150 20992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36241
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:11.010391 20992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:11.012166 20992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:11.024168 21004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:11.024449 20992 server_base.cc:1034] running on GCE node
W20260811 06:57:11.024544 21003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:11.024271 21006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:11.025609 20992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:11.026849 20992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:11.028050 20992 hybrid_clock.cc:648] HybridClock initialized: now 1786431431028001 us; error 46 us; skew 500 ppm
I20260811 06:57:11.028398 20992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:11.030591 20992 webserver.cc:460] Webserver started at http://127.4.40.62:35181/ using document root <none> and password file <none>
I20260811 06:57:11.031217 20992 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:11.031302 20992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:11.031575 20992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:11.034412 20992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "db825da429a346538e3971e909b1421c"
format_stamp: "Formatted at 2026-08-11 06:57:11 on dist-test-slave-7vjv"
server_key: "85b1c094353e80f283d83e85cc896a54"
server_key_iv: "816a832565ace0f25373227d560abbbc"
server_key_version: "encryptionkey@0"
I20260811 06:57:11.035135 20992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "db825da429a346538e3971e909b1421c"
format_stamp: "Formatted at 2026-08-11 06:57:11 on dist-test-slave-7vjv"
server_key: "85b1c094353e80f283d83e85cc896a54"
server_key_iv: "816a832565ace0f25373227d560abbbc"
server_key_version: "encryptionkey@0"
I20260811 06:57:11.040200 20992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 06:57:11.043421 21022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:11.044632 20992 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 06:57:11.044755 20992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "db825da429a346538e3971e909b1421c"
format_stamp: "Formatted at 2026-08-11 06:57:11 on dist-test-slave-7vjv"
server_key: "85b1c094353e80f283d83e85cc896a54"
server_key_iv: "816a832565ace0f25373227d560abbbc"
server_key_version: "encryptionkey@0"
I20260811 06:57:11.044914 20992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:11.079838 20992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:11.080703 20992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:11.080971 20992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:11.099684 20992 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:36241
I20260811 06:57:11.099706 21113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:36241 every 8 connection(s)
I20260811 06:57:11.101076 20992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:57:11.105480 21115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:57:11.105616 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 20992
I20260811 06:57:11.105796 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:57:11.106729 4256 external_mini_cluster.cc:1428] Setting key af9beabe1f14aad8a9f214afe6a3407e
I20260811 06:57:11.119956 21115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c: Bootstrap starting.
I20260811 06:57:11.121160 21120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:11.108882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.4.40.62:36241)
0811 06:57:11.109937 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:11.109948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:11.109989 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:57:11.111916 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:11.111938 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:11.112467 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:11.112820 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:11.113581 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:11.114752 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:11.116804 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:11.117426 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:11.117585 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:11.118931 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:11.118937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:11.118950 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:11.119458 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:11.119557 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:11.119590 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:11.119683 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:11.119941 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 06:57:11.120252 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":398,"threads_started":1}
I20260811 06:57:11.123368 21115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c: Neither blocks nor log segments found. Creating new log.
I20260811 06:57:11.124686 21115 log.cc:824] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:57:11.127502 21115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c: No bootstrap required, opened a new log
I20260811 06:57:11.135812 21115 raft_consensus.cc:348] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db825da429a346538e3971e909b1421c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36241 } }
I20260811 06:57:11.136258 21115 raft_consensus.cc:374] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:57:11.136404 21115 raft_consensus.cc:729] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db825da429a346538e3971e909b1421c, State: Initialized, Role: FOLLOWER
I20260811 06:57:11.137069 21115 consensus_queue.cc:260] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db825da429a346538e3971e909b1421c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36241 } }
I20260811 06:57:11.137338 21115 raft_consensus.cc:388] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:57:11.137501 21115 raft_consensus.cc:482] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:57:11.137719 21115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:57:11.140643 21115 raft_consensus.cc:504] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db825da429a346538e3971e909b1421c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36241 } }
I20260811 06:57:11.141255 21115 leader_election.cc:302] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: db825da429a346538e3971e909b1421c; no voters:
I20260811 06:57:11.141887 21115 leader_election.cc:288] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:57:11.141979 21122 raft_consensus.cc:2781] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:57:11.142836 21122 raft_consensus.cc:686] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [term 1 LEADER]: Becoming Leader. State: Replica: db825da429a346538e3971e909b1421c, State: Running, Role: LEADER
I20260811 06:57:11.143424 21122 consensus_queue.cc:237] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db825da429a346538e3971e909b1421c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36241 } }
I20260811 06:57:11.144323 21115 sys_catalog.cc:564] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:57:11.146394 21126 sys_catalog.cc:455] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [sys.catalog]: SysCatalogTable state changed. Reason: New leader db825da429a346538e3971e909b1421c. Latest consensus state: current_term: 1 leader_uuid: "db825da429a346538e3971e909b1421c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db825da429a346538e3971e909b1421c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36241 } } }
I20260811 06:57:11.146782 21126 sys_catalog.cc:458] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [sys.catalog]: This master's current role is: LEADER
I20260811 06:57:11.152031 21124 sys_catalog.cc:455] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "db825da429a346538e3971e909b1421c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "db825da429a346538e3971e909b1421c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36241 } } }
I20260811 06:57:11.152370 21124 sys_catalog.cc:458] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c [sys.catalog]: This master's current role is: LEADER
I20260811 06:57:11.153133 21133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:57:11.158258 21133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:57:11.166080 21133 catalog_manager.cc:1347] Generated new cluster ID: 9a857720e29c44198b01876aa6470ace
I20260811 06:57:11.166186 21133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:57:11.176153 21133 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:57:11.177815 21133 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:57:11.185253 21133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P db825da429a346538e3971e909b1421c: Generated new TSK 0
I20260811 06:57:11.186157 21133 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:57:11.200827 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36241
--builtin_ntp_servers=127.4.40.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 06:57:11.547287 21156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:11.547650 21156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:11.547756 21156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:11.554909 21156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:11.554992 21156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:11.555090 21156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:57:11.563707 21156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:11.564857 21156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:11.566526 21156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:11.579424 21164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:11.579372 21169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:11.579335 21166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:11.580793 21156 server_base.cc:1034] running on GCE node
I20260811 06:57:11.581384 21156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:11.582314 21156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:11.583545 21156 hybrid_clock.cc:648] HybridClock initialized: now 1786431431583505 us; error 49 us; skew 500 ppm
I20260811 06:57:11.583858 21156 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:11.589494 21156 webserver.cc:460] Webserver started at http://127.4.40.1:46303/ using document root <none> and password file <none>
I20260811 06:57:11.590374 21156 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:11.590543 21156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:11.591009 21156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:11.594156 21156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "3637f7c443dc42698eda4f404c0380e4"
format_stamp: "Formatted at 2026-08-11 06:57:11 on dist-test-slave-7vjv"
server_key: "a192f9958b5dedeb171e34077bc84487"
server_key_iv: "33e37269d723ce803886e1bfec8feb70"
server_key_version: "encryptionkey@0"
I20260811 06:57:11.595000 21156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "3637f7c443dc42698eda4f404c0380e4"
format_stamp: "Formatted at 2026-08-11 06:57:11 on dist-test-slave-7vjv"
server_key: "a192f9958b5dedeb171e34077bc84487"
server_key_iv: "33e37269d723ce803886e1bfec8feb70"
server_key_version: "encryptionkey@0"
I20260811 06:57:11.600260 21156 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 06:57:11.603639 21182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:11.605002 21156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:57:11.605194 21156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "3637f7c443dc42698eda4f404c0380e4"
format_stamp: "Formatted at 2026-08-11 06:57:11 on dist-test-slave-7vjv"
server_key: "a192f9958b5dedeb171e34077bc84487"
server_key_iv: "33e37269d723ce803886e1bfec8feb70"
server_key_version: "encryptionkey@0"
I20260811 06:57:11.605427 21156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:11.617355 21156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:11.618144 21156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:11.618436 21156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:11.619529 21156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:11.621217 21156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:11.621356 21156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:11.621490 21156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:11.621587 21156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:11.656769 21156 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:46855
I20260811 06:57:11.656826 21379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:46855 every 8 connection(s)
I20260811 06:57:11.658154 21156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:57:11.662295 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 21156
I20260811 06:57:11.662487 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:57:11.663132 4256 external_mini_cluster.cc:1428] Setting key 8bb8d3bfa177c7c13d341e2d51e26ead
I20260811 06:57:11.665952 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36241
--builtin_ntp_servers=127.4.40.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:57:11.670953 21382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:11.660731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:54239 (local address 127.4.40.62:36241)
0811 06:57:11.661210 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:11.661217 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:11.662412 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 06:57:11.664334 (+ 1922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:11.664338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:11.664399 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:11.664496 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:11.665734 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:11.666569 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:11.668156 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:11.668567 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:11.668705 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:11.670083 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:11.670088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:11.670091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:11.670110 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:11.670151 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:11.670162 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:11.670262 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:11.670487 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 06:57:11.670583 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":242,"threads_started":1}
I20260811 06:57:11.671916 21383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:11.661309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36241 (local address 127.4.40.1:54239)
0811 06:57:11.662248 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:11.662291 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:11.664016 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:11.664742 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:11.664751 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:11.664895 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:11.665559 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:11.665576 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:11.666734 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:11.666739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:11.667986 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:11.667999 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:11.668811 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:11.668818 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:11.669045 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:11.669741 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:57:11.669744 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:11.669785 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:57:11.669854 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:11.670263 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:11.670276 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:11.670286 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 06:57:11.670481 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 06:57:11.670851 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":299,"threads_started":1}
I20260811 06:57:11.673529 21380 heartbeater.cc:347] Connected to a master server at 127.4.40.62:36241
I20260811 06:57:11.673812 21380 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:11.674585 21380 heartbeater.cc:511] Master 127.4.40.62:36241 requested a full tablet report, sending...
I20260811 06:57:11.676855 21051 ts_manager.cc:194] Registered new tserver with Master: 3637f7c443dc42698eda4f404c0380e4 (127.4.40.1:46855)
I20260811 06:57:11.678947 21051 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.1:54239
W20260811 06:57:12.005354 21387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:12.005692 21387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:12.005865 21387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:12.013100 21387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:12.013262 21387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:12.013437 21387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:57:12.022393 21387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:12.023686 21387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:12.025516 21387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:12.038717 21398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:12.038786 21402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:12.038830 21404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:12.039388 21387 server_base.cc:1034] running on GCE node
I20260811 06:57:12.040000 21387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:12.041163 21387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:12.042361 21387 hybrid_clock.cc:648] HybridClock initialized: now 1786431432042317 us; error 38 us; skew 500 ppm
I20260811 06:57:12.042678 21387 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:12.044955 21387 webserver.cc:460] Webserver started at http://127.4.40.2:41175/ using document root <none> and password file <none>
I20260811 06:57:12.045696 21387 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:12.045814 21387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:12.046231 21387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:12.049196 21387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "d139c0524f264165a5fe3acfbbec7e11"
format_stamp: "Formatted at 2026-08-11 06:57:12 on dist-test-slave-7vjv"
server_key: "9191420e49cedc8f8a800c99a4bd22f8"
server_key_iv: "8f6eab039076c97dd41a6cce37cb4db2"
server_key_version: "encryptionkey@0"
I20260811 06:57:12.049995 21387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "d139c0524f264165a5fe3acfbbec7e11"
format_stamp: "Formatted at 2026-08-11 06:57:12 on dist-test-slave-7vjv"
server_key: "9191420e49cedc8f8a800c99a4bd22f8"
server_key_iv: "8f6eab039076c97dd41a6cce37cb4db2"
server_key_version: "encryptionkey@0"
I20260811 06:57:12.055251 21387 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:57:12.058646 21412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:12.059949 21387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 06:57:12.060153 21387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "d139c0524f264165a5fe3acfbbec7e11"
format_stamp: "Formatted at 2026-08-11 06:57:12 on dist-test-slave-7vjv"
server_key: "9191420e49cedc8f8a800c99a4bd22f8"
server_key_iv: "8f6eab039076c97dd41a6cce37cb4db2"
server_key_version: "encryptionkey@0"
I20260811 06:57:12.060359 21387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:12.130376 21387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:12.131253 21387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:12.131546 21387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:12.132551 21387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:12.134052 21387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:12.134151 21387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:12.134310 21387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:12.134402 21387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:12.168452 21387 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:33649
I20260811 06:57:12.168509 21611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:33649 every 8 connection(s)
I20260811 06:57:12.169765 21387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:57:12.179421 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 21387
I20260811 06:57:12.179608 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:57:12.180073 4256 external_mini_cluster.cc:1428] Setting key bbbb682463e4f6a5a0aa26b38e9708d2
I20260811 06:57:12.182547 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.172372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:34573 (local address 127.4.40.62:36241)
0811 06:57:12.172840 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:12.172844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:12.174040 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 06:57:12.175830 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:12.175835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:12.175928 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:12.176058 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:12.177150 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.177904 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.179442 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.179968 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.180128 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.181658 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:12.181663 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:12.181666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:12.181684 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:12.181724 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:12.181736 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:12.181815 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:12.182107 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 06:57:12.182200 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":255,"threads_started":1}
I20260811 06:57:12.183189 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36241
--builtin_ntp_servers=127.4.40.20:33451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 06:57:12.183568 21617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.172988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36241 (local address 127.4.40.2:34573)
0811 06:57:12.173874 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:12.173918 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:12.175554 (+ 1636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:12.176302 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:12.176310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:12.176395 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:12.176973 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:12.176991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.178078 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.178082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:12.179296 (+ 1214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:12.179310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.180228 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.180237 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:12.180469 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.181310 (+ 841us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:57:12.181313 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:12.181361 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:57:12.181466 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:12.181851 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:12.181862 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:12.181870 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:57:12.182076 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 06:57:12.182456 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":281,"threads_started":1}
I20260811 06:57:12.185051 21612 heartbeater.cc:347] Connected to a master server at 127.4.40.62:36241
I20260811 06:57:12.185385 21612 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:12.186132 21612 heartbeater.cc:511] Master 127.4.40.62:36241 requested a full tablet report, sending...
I20260811 06:57:12.187577 21051 ts_manager.cc:194] Registered new tserver with Master: d139c0524f264165a5fe3acfbbec7e11 (127.4.40.2:33649)
I20260811 06:57:12.188432 21051 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.2:34573
W20260811 06:57:12.530570 21620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:12.530864 21620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:12.530953 21620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:12.538650 21620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:12.538751 21620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:12.538879 21620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:57:12.547976 21620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:36241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:12.549280 21620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:12.551064 21620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:12.564862 21631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:12.564946 21629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:12.564930 21633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:12.566215 21620 server_base.cc:1034] running on GCE node
I20260811 06:57:12.566815 21620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:12.568121 21620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:12.569361 21620 hybrid_clock.cc:648] HybridClock initialized: now 1786431432569303 us; error 53 us; skew 500 ppm
I20260811 06:57:12.569702 21620 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:12.578152 21620 webserver.cc:460] Webserver started at http://127.4.40.3:35681/ using document root <none> and password file <none>
I20260811 06:57:12.578938 21620 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:12.579097 21620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:12.579450 21620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:12.582487 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "f6c2914bb3094a9eaaa2d8bdd1326c5a"
format_stamp: "Formatted at 2026-08-11 06:57:12 on dist-test-slave-7vjv"
server_key: "a710d63955e4daaa034ad621f25d0dc1"
server_key_iv: "6f586c01b75ebcecfe62ae97ef61a018"
server_key_version: "encryptionkey@0"
I20260811 06:57:12.583334 21620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "f6c2914bb3094a9eaaa2d8bdd1326c5a"
format_stamp: "Formatted at 2026-08-11 06:57:12 on dist-test-slave-7vjv"
server_key: "a710d63955e4daaa034ad621f25d0dc1"
server_key_iv: "6f586c01b75ebcecfe62ae97ef61a018"
server_key_version: "encryptionkey@0"
I20260811 06:57:12.588464 21620 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 06:57:12.591990 21645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:12.593343 21620 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:57:12.593544 21620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "f6c2914bb3094a9eaaa2d8bdd1326c5a"
format_stamp: "Formatted at 2026-08-11 06:57:12 on dist-test-slave-7vjv"
server_key: "a710d63955e4daaa034ad621f25d0dc1"
server_key_iv: "6f586c01b75ebcecfe62ae97ef61a018"
server_key_version: "encryptionkey@0"
I20260811 06:57:12.593763 21620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:12.607276 21620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:12.608080 21620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:12.608424 21620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:12.609342 21620 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:12.610891 21620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:12.611020 21620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:12.611164 21620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:12.611256 21620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:12.646031 21620 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:41995
I20260811 06:57:12.646086 21849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:41995 every 8 connection(s)
I20260811 06:57:12.647415 21620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:57:12.654124 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 21620
I20260811 06:57:12.654325 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:57:12.654891 4256 external_mini_cluster.cc:1428] Setting key 8d3afc137fcef0802960fc0bd87727eb
I20260811 06:57:12.660895 21616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.650067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:55367 (local address 127.4.40.62:36241)
0811 06:57:12.650287 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:12.650293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:12.651659 (+ 1366us) server_negotiation.cc:407] Connection header received
0811 06:57:12.653407 (+ 1748us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:12.653412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:12.653480 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:12.653569 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:12.654729 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.655729 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.658094 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.658496 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.658589 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.660198 (+ 1609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:12.660203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:12.660207 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:12.660225 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:12.660255 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:12.660269 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:12.660337 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:12.660604 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 06:57:12.660681 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:57:12.662192 21856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.650605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36241 (local address 127.4.40.3:55367)
0811 06:57:12.651504 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:12.651539 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:12.653154 (+ 1615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:12.653780 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:12.653787 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:12.653864 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:12.654559 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:12.654578 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.655915 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.655924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:12.657924 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:12.657939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.658713 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.658720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:12.659003 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.659759 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:57:12.659763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:12.659816 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:57:12.659906 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:12.660371 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:12.660385 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:12.660396 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 06:57:12.660586 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 06:57:12.660935 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":306,"threads_started":1}
I20260811 06:57:12.663614 21851 heartbeater.cc:347] Connected to a master server at 127.4.40.62:36241
I20260811 06:57:12.663916 21851 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:12.664844 21851 heartbeater.cc:511] Master 127.4.40.62:36241 requested a full tablet report, sending...
I20260811 06:57:12.666328 21051 ts_manager.cc:194] Registered new tserver with Master: f6c2914bb3094a9eaaa2d8bdd1326c5a (127.4.40.3:41995)
I20260811 06:57:12.667125 21051 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.3:55367
I20260811 06:57:12.670279 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:57:12.676770 21616 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:12.675019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.4.40.62:36241)
0811 06:57:12.675216 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:12.675224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:12.675653 (+ 429us) server_negotiation.cc:407] Connection header received
0811 06:57:12.675842 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:12.675848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:12.675914 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:12.676019 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:12.676498 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59754: BlockingRecv error: recv got EOF from 127.0.0.1:59754 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:57:12.677547 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 21156
I20260811 06:57:12.682451 21380 heartbeater.cc:503] Master 127.4.40.62:36241 was elected leader, sending a full tablet report...
I20260811 06:57:12.685349 21871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.678457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42264 (local address 127.4.40.1:46855)
0811 06:57:12.679007 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:12.679013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:12.679037 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:57:12.679087 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:12.679092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:12.679281 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:12.679398 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:12.679753 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.680816 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.681982 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.682719 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.682830 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.683981 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:12.683991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:12.684003 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:12.684644 (+ 641us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:12.684689 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:12.684707 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:12.684793 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:12.684947 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 06:57:12.685050 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":274,"threads_started":1}
I20260811 06:57:12.686508 21374 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:12.790594 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 21156
I20260811 06:57:12.817716 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 21387
I20260811 06:57:12.825718 21875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.818573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.4.40.2:33649)
0811 06:57:12.819123 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:12.819130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:12.819157 (+ 27us) server_negotiation.cc:407] Connection header received
0811 06:57:12.819218 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:12.819224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:12.819430 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:12.819641 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:12.820129 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.821170 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.822426 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.822953 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.823084 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.824226 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:12.824240 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:12.824257 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:12.824861 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:12.824908 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:12.824927 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:12.825060 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:12.825215 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 06:57:12.825342 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":265,"threads_started":1}
I20260811 06:57:12.826931 21602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:12.908744 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 21387
I20260811 06:57:12.934293 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 21620
I20260811 06:57:12.943111 21882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:12.935327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.4.40.3:41995)
0811 06:57:12.935983 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:12.935990 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:12.936024 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:57:12.936137 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:12.936148 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:12.936405 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:12.936624 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:12.937133 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.938184 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.939454 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:12.940066 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:12.940367 (+ 301us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:12.941446 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:12.941454 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:12.941473 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:12.942135 (+ 662us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:12.942183 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:12.942210 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:12.942413 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:12.942571 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 06:57:12.942729 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":337,"threads_started":1}
I20260811 06:57:12.944468 21845 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:13.026407 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 21620
I20260811 06:57:13.054301 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 20992
I20260811 06:57:13.055524 21107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:13.157809 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 20992
2026-08-11T06:57:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2602 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T06:57:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:57:13Z Disabled control of system clock
I20260811 06:57:13.253564 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:39045
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:39045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:57:13.593253 21890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:13.593537 21890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:13.593621 21890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:13.600502 21890 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:57:13.600590 21890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:13.600692 21890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:13.600811 21890 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:57:13.600884 21890 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:57:13.609786 21890 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:39045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:39045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:13.611052 21890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:13.612851 21890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:13.624982 21910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:13.625102 21906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:13.625154 21904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:13.626075 21890 server_base.cc:1034] running on GCE node
I20260811 06:57:13.626816 21890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:13.628273 21890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:13.629498 21890 hybrid_clock.cc:648] HybridClock initialized: now 1786431433629412 us; error 76 us; skew 500 ppm
I20260811 06:57:13.629846 21890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:13.632040 21890 webserver.cc:460] Webserver started at http://127.4.40.62:35821/ using document root <none> and password file <none>
I20260811 06:57:13.632745 21890 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:13.632877 21890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:13.633213 21890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:13.636032 21890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "622ddf500b39435b8816e962cdab2d8d"
format_stamp: "Formatted at 2026-08-11 06:57:13 on dist-test-slave-7vjv"
server_key: "f627e208a2d382fe5fd750bde5fb36dd"
server_key_iv: "0e6966aed5ea487169bd37a31dab1c27"
server_key_version: "encryptionkey@0"
I20260811 06:57:13.636862 21890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "622ddf500b39435b8816e962cdab2d8d"
format_stamp: "Formatted at 2026-08-11 06:57:13 on dist-test-slave-7vjv"
server_key: "f627e208a2d382fe5fd750bde5fb36dd"
server_key_iv: "0e6966aed5ea487169bd37a31dab1c27"
server_key_version: "encryptionkey@0"
I20260811 06:57:13.641975 21890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:57:13.645407 21921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:13.646699 21890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:57:13.646904 21890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "622ddf500b39435b8816e962cdab2d8d"
format_stamp: "Formatted at 2026-08-11 06:57:13 on dist-test-slave-7vjv"
server_key: "f627e208a2d382fe5fd750bde5fb36dd"
server_key_iv: "0e6966aed5ea487169bd37a31dab1c27"
server_key_version: "encryptionkey@0"
I20260811 06:57:13.647137 21890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:13.652946 21890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:13.653661 21890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:13.653955 21890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:13.672055 21890 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:39045
I20260811 06:57:13.672148 21995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:39045 every 8 connection(s)
I20260811 06:57:13.673350 21890 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:57:13.676889 21996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:57:13.679750 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 21890
I20260811 06:57:13.679991 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:57:13.680660 4256 external_mini_cluster.cc:1428] Setting key dc0dc82288f9a8d475fd7a97cfd11cf7
I20260811 06:57:13.688764 22001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:13.682626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40266 (local address 127.4.40.62:39045)
0811 06:57:13.683566 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:13.683575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:13.683614 (+ 39us) server_negotiation.cc:407] Connection header received
0811 06:57:13.685274 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:13.685296 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:13.685698 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:13.686082 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:13.687169 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:13.687175 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:13.687186 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:13.687199 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:13.687255 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:13.687269 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:13.687500 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:13.687647 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 06:57:13.687871 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":367,"threads_started":1}
I20260811 06:57:13.688849 21996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d: Bootstrap starting.
I20260811 06:57:13.691913 21996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d: Neither blocks nor log segments found. Creating new log.
I20260811 06:57:13.692978 21996 log.cc:824] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d: Log is configured to *not* fsync() on all Append() calls
I20260811 06:57:13.695490 21996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d: No bootstrap required, opened a new log
I20260811 06:57:13.702606 21996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622ddf500b39435b8816e962cdab2d8d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39045 } }
I20260811 06:57:13.702932 21996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:57:13.703054 21996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622ddf500b39435b8816e962cdab2d8d, State: Initialized, Role: FOLLOWER
I20260811 06:57:13.703583 21996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622ddf500b39435b8816e962cdab2d8d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39045 } }
I20260811 06:57:13.703804 21996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:57:13.703934 21996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:57:13.704119 21996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:57:13.706208 21996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622ddf500b39435b8816e962cdab2d8d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39045 } }
I20260811 06:57:13.706691 21996 leader_election.cc:302] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 622ddf500b39435b8816e962cdab2d8d; no voters:
I20260811 06:57:13.707329 21996 leader_election.cc:288] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:57:13.707479 22003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:57:13.708284 22003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [term 1 LEADER]: Becoming Leader. State: Replica: 622ddf500b39435b8816e962cdab2d8d, State: Running, Role: LEADER
I20260811 06:57:13.708832 22003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622ddf500b39435b8816e962cdab2d8d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39045 } }
I20260811 06:57:13.709486 21996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:57:13.711856 22004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "622ddf500b39435b8816e962cdab2d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622ddf500b39435b8816e962cdab2d8d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39045 } } }
I20260811 06:57:13.711856 22009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 622ddf500b39435b8816e962cdab2d8d. Latest consensus state: current_term: 1 leader_uuid: "622ddf500b39435b8816e962cdab2d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "622ddf500b39435b8816e962cdab2d8d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 39045 } } }
I20260811 06:57:13.712332 22004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [sys.catalog]: This master's current role is: LEADER
I20260811 06:57:13.712461 22009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d [sys.catalog]: This master's current role is: LEADER
I20260811 06:57:13.714172 22020 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:57:13.719175 22020 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:57:13.727897 22020 catalog_manager.cc:1347] Generated new cluster ID: 44836d4d6d29498ca0c78f3ef01b52af
I20260811 06:57:13.727995 22020 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:57:13.757742 22020 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:57:13.758224 22020 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:57:13.770432 22020 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 622ddf500b39435b8816e962cdab2d8d: Generated new TSK 0
I20260811 06:57:13.771275 22020 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:57:13.786942 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:39045
--builtin_ntp_servers=127.4.40.20:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 06:57:14.133200 22043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:14.133483 22043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:14.133657 22043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:14.140691 22043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:14.140808 22043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:14.140993 22043 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:57:14.150298 22043 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:39045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:14.151662 22043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:14.153589 22043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:14.166865 22065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:14.166865 22059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:14.166985 22058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:14.168181 22043 server_base.cc:1034] running on GCE node
I20260811 06:57:14.168859 22043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:14.169772 22043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:14.171011 22043 hybrid_clock.cc:648] HybridClock initialized: now 1786431434170950 us; error 65 us; skew 500 ppm
I20260811 06:57:14.171335 22043 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:14.173555 22043 webserver.cc:460] Webserver started at http://127.4.40.1:46295/ using document root <none> and password file <none>
I20260811 06:57:14.174253 22043 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:14.174386 22043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:14.174722 22043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:14.177647 22043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "8c74c16fc64642749cc6313912422fb3"
format_stamp: "Formatted at 2026-08-11 06:57:14 on dist-test-slave-7vjv"
server_key: "58486604f248a3a1a211ecc5d04f6229"
server_key_iv: "65bc5ac111d91628ddeeec3751b40ef0"
server_key_version: "encryptionkey@0"
I20260811 06:57:14.178437 22043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "8c74c16fc64642749cc6313912422fb3"
format_stamp: "Formatted at 2026-08-11 06:57:14 on dist-test-slave-7vjv"
server_key: "58486604f248a3a1a211ecc5d04f6229"
server_key_iv: "65bc5ac111d91628ddeeec3751b40ef0"
server_key_version: "encryptionkey@0"
I20260811 06:57:14.183666 22043 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 06:57:14.187067 22075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:14.188694 22043 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:57:14.188887 22043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "8c74c16fc64642749cc6313912422fb3"
format_stamp: "Formatted at 2026-08-11 06:57:14 on dist-test-slave-7vjv"
server_key: "58486604f248a3a1a211ecc5d04f6229"
server_key_iv: "65bc5ac111d91628ddeeec3751b40ef0"
server_key_version: "encryptionkey@0"
I20260811 06:57:14.189119 22043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:14.194775 22043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:14.195478 22043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:14.195712 22043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:14.196624 22043 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:14.197961 22043 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:14.198045 22043 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:14.198122 22043 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:14.198175 22043 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:14.231813 22043 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:33481
I20260811 06:57:14.231873 22286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:33481 every 8 connection(s)
I20260811 06:57:14.233230 22043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:57:14.237790 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 22043
I20260811 06:57:14.238013 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:57:14.238803 4256 external_mini_cluster.cc:1428] Setting key 72624c2ed862898b883bc6effa654803
I20260811 06:57:14.242202 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:39045
--builtin_ntp_servers=127.4.40.20:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:57:14.242271 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:14.235789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57241 (local address 127.4.40.62:39045)
0811 06:57:14.236233 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:14.236238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:14.237409 (+ 1171us) server_negotiation.cc:407] Connection header received
0811 06:57:14.239768 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:14.239774 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:14.239835 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:14.239952 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:14.241461 (+ 1509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:14.241467 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:14.241471 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:14.241491 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:14.241543 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:14.241559 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:14.241690 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:14.241862 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 06:57:14.241928 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260811 06:57:14.243067 22292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:14.236329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:39045 (local address 127.4.40.1:57241)
0811 06:57:14.237214 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:14.237281 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:14.239505 (+ 2224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:14.240239 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:14.240250 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:14.240360 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:14.241160 (+ 800us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:57:14.241163 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:14.241200 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:57:14.241284 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:14.241664 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:14.241673 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:14.241678 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 06:57:14.241883 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 06:57:14.242065 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":281,"threads_started":1}
I20260811 06:57:14.244535 22287 heartbeater.cc:347] Connected to a master server at 127.4.40.62:39045
I20260811 06:57:14.244827 22287 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:14.245364 22287 heartbeater.cc:511] Master 127.4.40.62:39045 requested a full tablet report, sending...
I20260811 06:57:14.247345 21942 ts_manager.cc:194] Registered new tserver with Master: 8c74c16fc64642749cc6313912422fb3 (127.4.40.1:33481)
W20260811 06:57:14.591499 22295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:14.591874 22295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:14.592058 22295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:14.599821 22295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:14.599953 22295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:14.600172 22295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:57:14.608994 22295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:39045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:14.610374 22295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:14.612329 22295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:14.625501 22301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:14.625468 22303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:14.625527 22307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:14.626539 22295 server_base.cc:1034] running on GCE node
I20260811 06:57:14.627195 22295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:14.628221 22295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:14.629460 22295 hybrid_clock.cc:648] HybridClock initialized: now 1786431434629418 us; error 50 us; skew 500 ppm
I20260811 06:57:14.629789 22295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:14.632211 22295 webserver.cc:460] Webserver started at http://127.4.40.2:37691/ using document root <none> and password file <none>
I20260811 06:57:14.632926 22295 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:14.633059 22295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:14.633400 22295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:14.636318 22295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "d80b6daf0a8d44cda545a03f231a37bd"
format_stamp: "Formatted at 2026-08-11 06:57:14 on dist-test-slave-7vjv"
server_key: "eb7d9cddb32a305feab85444e418cfdd"
server_key_iv: "d00c6a7dea98f52796448fc4b0b65d0f"
server_key_version: "encryptionkey@0"
I20260811 06:57:14.637104 22295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "d80b6daf0a8d44cda545a03f231a37bd"
format_stamp: "Formatted at 2026-08-11 06:57:14 on dist-test-slave-7vjv"
server_key: "eb7d9cddb32a305feab85444e418cfdd"
server_key_iv: "d00c6a7dea98f52796448fc4b0b65d0f"
server_key_version: "encryptionkey@0"
I20260811 06:57:14.642169 22295 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:57:14.645648 22316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:14.646929 22295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:57:14.647105 22295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "d80b6daf0a8d44cda545a03f231a37bd"
format_stamp: "Formatted at 2026-08-11 06:57:14 on dist-test-slave-7vjv"
server_key: "eb7d9cddb32a305feab85444e418cfdd"
server_key_iv: "d00c6a7dea98f52796448fc4b0b65d0f"
server_key_version: "encryptionkey@0"
I20260811 06:57:14.647323 22295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:14.653241 22295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:14.654003 22295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:14.654273 22295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:14.655160 22295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:14.656615 22295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:14.656737 22295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:14.656889 22295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:14.656994 22295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:14.693395 22295 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:34045
I20260811 06:57:14.693454 22518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:34045 every 8 connection(s)
I20260811 06:57:14.694763 22295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:57:14.702975 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:14.697492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:54203 (local address 127.4.40.62:39045)
0811 06:57:14.697660 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:14.697664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:14.698998 (+ 1334us) server_negotiation.cc:407] Connection header received
0811 06:57:14.700784 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:14.700789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:14.700869 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:14.700996 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:14.702218 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:14.702224 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:14.702228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:14.702247 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:14.702287 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:14.702300 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:14.702436 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:14.702638 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 06:57:14.702707 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:57:14.703716 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 22295
I20260811 06:57:14.703912 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:57:14.703902 22523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:14.697941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:39045 (local address 127.4.40.2:54203)
0811 06:57:14.698846 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:14.698879 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:14.700525 (+ 1646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:14.701211 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:14.701218 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:14.701318 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:14.701931 (+ 613us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:57:14.701933 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:14.701970 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:57:14.702048 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:14.702411 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:14.702423 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:14.702431 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 06:57:14.702712 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 06:57:14.702938 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":302,"threads_started":1}
I20260811 06:57:14.704403 4256 external_mini_cluster.cc:1428] Setting key c157b6f799001a75c0927e6ece32e5f7
I20260811 06:57:14.705384 22519 heartbeater.cc:347] Connected to a master server at 127.4.40.62:39045
I20260811 06:57:14.705703 22519 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:14.706384 22519 heartbeater.cc:511] Master 127.4.40.62:39045 requested a full tablet report, sending...
I20260811 06:57:14.707584 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:39045
--builtin_ntp_servers=127.4.40.20:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 06:57:14.707952 21942 ts_manager.cc:194] Registered new tserver with Master: d80b6daf0a8d44cda545a03f231a37bd (127.4.40.2:34045)
W20260811 06:57:15.047775 22528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:15.048135 22528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:15.048343 22528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:15.055956 22528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:15.056100 22528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:15.056315 22528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:57:15.065448 22528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:39045
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:15.066814 22528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:15.068770 22528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:15.082594 22546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:15.082719 22535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:15.082685 22538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:15.083562 22528 server_base.cc:1034] running on GCE node
I20260811 06:57:15.084278 22528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:15.085203 22528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:15.086438 22528 hybrid_clock.cc:648] HybridClock initialized: now 1786431435086392 us; error 53 us; skew 500 ppm
I20260811 06:57:15.086755 22528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 06:57:15.089035 22528 webserver.cc:460] Webserver started at http://127.4.40.3:45155/ using document root <none> and password file <none>
I20260811 06:57:15.089747 22528 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:15.089885 22528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:15.090233 22528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:15.093183 22528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "2c066d12c71c41d5b8e8a4244aec050f"
format_stamp: "Formatted at 2026-08-11 06:57:15 on dist-test-slave-7vjv"
server_key: "9137cddb7c53039f42bebf8400816a1d"
server_key_iv: "fe12057f7724dec39b5194503b4015cb"
server_key_version: "encryptionkey@0"
I20260811 06:57:15.093973 22528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "2c066d12c71c41d5b8e8a4244aec050f"
format_stamp: "Formatted at 2026-08-11 06:57:15 on dist-test-slave-7vjv"
server_key: "9137cddb7c53039f42bebf8400816a1d"
server_key_iv: "fe12057f7724dec39b5194503b4015cb"
server_key_version: "encryptionkey@0"
I20260811 06:57:15.098886 22528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 06:57:15.102228 22557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:15.103461 22528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:57:15.103643 22528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "2c066d12c71c41d5b8e8a4244aec050f"
format_stamp: "Formatted at 2026-08-11 06:57:15 on dist-test-slave-7vjv"
server_key: "9137cddb7c53039f42bebf8400816a1d"
server_key_iv: "fe12057f7724dec39b5194503b4015cb"
server_key_version: "encryptionkey@0"
I20260811 06:57:15.103866 22528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:15.109432 22528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:15.110318 22528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:15.110661 22528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:15.111564 22528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:15.113022 22528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:15.113147 22528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:15.113277 22528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:15.113371 22528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:15.147194 22528 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:46629
I20260811 06:57:15.147286 22774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:46629 every 8 connection(s)
I20260811 06:57:15.148748 22528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:57:15.156894 22291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:15.151516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:45483 (local address 127.4.40.62:39045)
0811 06:57:15.151740 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:15.151746 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:15.153262 (+ 1516us) server_negotiation.cc:407] Connection header received
0811 06:57:15.154853 (+ 1591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:15.154857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:15.154909 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:15.155000 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:15.156319 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:15.156323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:15.156326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:15.156340 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:15.156371 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:15.156382 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:15.156472 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:15.156634 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 06:57:15.156684 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:57:15.157768 22777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:15.152198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:39045 (local address 127.4.40.3:45483)
0811 06:57:15.153106 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:15.153143 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:15.154629 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:15.155197 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:15.155204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:15.155284 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:15.155978 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 06:57:15.155981 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:15.156030 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 06:57:15.156175 (+ 145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:15.156466 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:15.156473 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:15.156479 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 06:57:15.156680 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 06:57:15.156856 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":283,"threads_started":1}
I20260811 06:57:15.158805 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 22528
I20260811 06:57:15.159049 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:57:15.159161 22775 heartbeater.cc:347] Connected to a master server at 127.4.40.62:39045
I20260811 06:57:15.159484 22775 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:15.159678 4256 external_mini_cluster.cc:1428] Setting key bb1de7f1567929b5689495ae2aab4037
I20260811 06:57:15.160259 22775 heartbeater.cc:511] Master 127.4.40.62:39045 requested a full tablet report, sending...
I20260811 06:57:15.161880 21942 ts_manager.cc:194] Registered new tserver with Master: 2c066d12c71c41d5b8e8a4244aec050f (127.4.40.3:46629)
I20260811 06:57:15.163313 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 06:57:15.169736 22291 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 06:57:15.168204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40280 (local address 127.4.40.62:39045)
0811 06:57:15.168388 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:15.168396 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:15.168934 (+ 538us) server_negotiation.cc:407] Connection header received
0811 06:57:15.169059 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:15.169063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:15.169109 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:15.169183 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:15.169508 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40280: BlockingRecv error: recv got EOF from 127.0.0.1:40280 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 06:57:15.170574 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 22043
I20260811 06:57:15.174232 22787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:15.171481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47692 (local address 127.4.40.1:33481)
0811 06:57:15.172008 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:15.172015 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:15.172040 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:57:15.172142 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:15.172150 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:15.172336 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:15.172494 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:15.173487 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:15.173493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:15.173502 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:15.173527 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:15.173563 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:15.173576 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:15.173751 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:15.173873 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 06:57:15.173981 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":238,"threads_started":1}
I20260811 06:57:15.175347 22279 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:15.255117 22287 heartbeater.cc:503] Master 127.4.40.62:39045 was elected leader, sending a full tablet report...
I20260811 06:57:15.260277 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 22043
I20260811 06:57:15.286367 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 22295
I20260811 06:57:15.290135 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:15.287448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34194 (local address 127.4.40.2:34045)
0811 06:57:15.288051 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:15.288064 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:15.288144 (+ 80us) server_negotiation.cc:407] Connection header received
0811 06:57:15.288221 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:15.288228 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:15.288447 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:15.288579 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:15.289456 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:15.289463 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:15.289475 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:15.289495 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:15.289541 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:15.289556 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:15.289662 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:15.289790 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 06:57:15.289873 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":298,"threads_started":1}
I20260811 06:57:15.291354 22513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:15.377442 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 22295
I20260811 06:57:15.405346 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 22528
I20260811 06:57:15.409358 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:15.406258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58358 (local address 127.4.40.3:46629)
0811 06:57:15.406897 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:15.406904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:15.406936 (+ 32us) server_negotiation.cc:407] Connection header received
0811 06:57:15.407000 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:15.407028 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:15.407248 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:15.407470 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:15.408588 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:15.408605 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:15.408615 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 06:57:15.408626 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:15.408664 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 06:57:15.408676 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:15.408802 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:15.408849 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 06:57:15.408967 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":308,"threads_started":1}
I20260811 06:57:15.410674 22764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:15.492826 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 22528
I20260811 06:57:15.523675 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 21890
I20260811 06:57:15.524981 21988 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:15.627454 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 21890
2026-08-11T06:57:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2468 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:57:15 dist-test-slave-7vjv krb5kdc[22801](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:57:15 dist-test-slave-7vjv krb5kdc[22801](info): set up 2 sockets
Aug 11 06:57:15 dist-test-slave-7vjv krb5kdc[22801](info): commencing operation
krb5kdc: starting...
W20260811 06:57:16.996685 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.318s user 0.007s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:57:17 dist-test-slave-7vjv krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:57:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:57:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:57:17.283449 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:41667
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:41667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:57:17.621176 22823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:17.621433 22823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:17.621526 22823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:17.628638 22823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:57:17.628739 22823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:17.628841 22823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:17.628952 22823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:57:17.629019 22823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:57:17.638824 22823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:41667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:41667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:17.640220 22823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:17.641623 22823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:17.653676 22831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:17.653640 22832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:17.653738 22834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:17.654984 22823 server_base.cc:1034] running on GCE node
I20260811 06:57:17.655761 22823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:17.657212 22823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:17.658432 22823 hybrid_clock.cc:648] HybridClock initialized: now 1786431437658388 us; error 34 us; skew 500 ppm
I20260811 06:57:17.658775 22823 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:57:17 dist-test-slave-7vjv krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431437, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:57:17.665935 22823 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:57:17.667342 22823 webserver.cc:460] Webserver started at http://127.4.40.62:34017/ using document root <none> and password file <none>
I20260811 06:57:17.667996 22823 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:17.668149 22823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:17.668454 22823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:17.671352 22823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "b6627d46ded949a99797cb449146fb99"
format_stamp: "Formatted at 2026-08-11 06:57:17 on dist-test-slave-7vjv"
server_key: "ec564e9807b41ee480d25d15eafe1905"
server_key_iv: "01b992d6d9f7f74e6fdc29f243367267"
server_key_version: "encryptionkey@0"
I20260811 06:57:17.672169 22823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "b6627d46ded949a99797cb449146fb99"
format_stamp: "Formatted at 2026-08-11 06:57:17 on dist-test-slave-7vjv"
server_key: "ec564e9807b41ee480d25d15eafe1905"
server_key_iv: "01b992d6d9f7f74e6fdc29f243367267"
server_key_version: "encryptionkey@0"
I20260811 06:57:17.677295 22823 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:57:17.681056 22853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:17.682456 22823 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:57:17.682602 22823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "b6627d46ded949a99797cb449146fb99"
format_stamp: "Formatted at 2026-08-11 06:57:17 on dist-test-slave-7vjv"
server_key: "ec564e9807b41ee480d25d15eafe1905"
server_key_iv: "01b992d6d9f7f74e6fdc29f243367267"
server_key_version: "encryptionkey@0"
I20260811 06:57:17.682760 22823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:17.701839 22823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:17.705741 22823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:17.705983 22823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:17.724831 22823 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:41667
I20260811 06:57:17.724900 22947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:41667 every 8 connection(s)
I20260811 06:57:17.726174 22823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:57:17.729738 22948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:57:17.731602 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 22823
I20260811 06:57:17.731856 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:57:17.732743 4256 external_mini_cluster.cc:1428] Setting key c67c64b22d9e34ceaaf8773fc0d4332f
I20260811 06:57:17.741643 22948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99: Bootstrap starting.
I20260811 06:57:17.744805 22948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99: Neither blocks nor log segments found. Creating new log.
I20260811 06:57:17.745848 22948 log.cc:824] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99: Log is configured to *not* fsync() on all Append() calls
Aug 11 06:57:17 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:57:17.749136 22948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99: No bootstrap required, opened a new log
I20260811 06:57:17.755498 22959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:17.735067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60722 (local address 127.4.40.62:41667)
0811 06:57:17.735834 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:17.735845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:17.735887 (+ 42us) server_negotiation.cc:407] Connection header received
0811 06:57:17.737601 (+ 1714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:17.737620 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:17.737948 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:17.738300 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:17.739826 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:17.741171 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:17.742541 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:17.743217 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:17.743416 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:17.747961 (+ 4545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:17.747970 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:17.747984 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:17.748022 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:17.751208 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:17.752083 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:17.752089 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:17.752140 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:17.752319 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:17.752737 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:17.752740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:17.752742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:17.753332 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:17.753571 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:17.754043 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 06:57:17.754389 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":339,"threads_started":1}
I20260811 06:57:17.758244 22948 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6627d46ded949a99797cb449146fb99" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41667 } }
I20260811 06:57:17.758584 22948 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:57:17.758690 22948 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6627d46ded949a99797cb449146fb99, State: Initialized, Role: FOLLOWER
I20260811 06:57:17.759210 22948 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6627d46ded949a99797cb449146fb99" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41667 } }
I20260811 06:57:17.759485 22948 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:57:17.759663 22948 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:57:17.759840 22948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:57:17.762007 22948 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6627d46ded949a99797cb449146fb99" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41667 } }
I20260811 06:57:17.762482 22948 leader_election.cc:302] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6627d46ded949a99797cb449146fb99; no voters:
I20260811 06:57:17.763192 22948 leader_election.cc:288] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:57:17.763304 22961 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:57:17.763998 22961 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [term 1 LEADER]: Becoming Leader. State: Replica: b6627d46ded949a99797cb449146fb99, State: Running, Role: LEADER
I20260811 06:57:17.764662 22961 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6627d46ded949a99797cb449146fb99" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41667 } }
I20260811 06:57:17.765457 22948 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:57:17.767514 22963 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6627d46ded949a99797cb449146fb99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6627d46ded949a99797cb449146fb99" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41667 } } }
I20260811 06:57:17.768126 22963 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [sys.catalog]: This master's current role is: LEADER
I20260811 06:57:17.768131 22964 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6627d46ded949a99797cb449146fb99. Latest consensus state: current_term: 1 leader_uuid: "b6627d46ded949a99797cb449146fb99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6627d46ded949a99797cb449146fb99" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 41667 } } }
I20260811 06:57:17.770565 22964 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99 [sys.catalog]: This master's current role is: LEADER
I20260811 06:57:17.770682 22977 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:57:17.775765 22977 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:57:17.784387 22977 catalog_manager.cc:1347] Generated new cluster ID: c1f3c08cbec748b989deef2d0a842a69
I20260811 06:57:17.784516 22977 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:57:17.794751 22977 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:57:17.795914 22977 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:57:17.803231 22977 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99: Generated new TSK 0
I20260811 06:57:17.804086 22977 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:57:17.880817 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:41667
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:57:18.220643 22992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:18.220903 22992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:18.220997 22992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:18.228004 22992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:18.228150 22992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:18.228283 22992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:57:18.237530 22992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:41667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:18.238642 22992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:18.239928 22992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:18.253615 23004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:18.253731 23003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:18.253726 23008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:18.254565 22992 server_base.cc:1034] running on GCE node
I20260811 06:57:18.255205 22992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:18.256242 22992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:18.257462 22992 hybrid_clock.cc:648] HybridClock initialized: now 1786431438257424 us; error 48 us; skew 500 ppm
I20260811 06:57:18.257778 22992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:57:18 dist-test-slave-7vjv krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:57:18.261198 22992 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:57:18.262544 22992 webserver.cc:460] Webserver started at http://127.4.40.1:43141/ using document root <none> and password file <none>
I20260811 06:57:18.263240 22992 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:18.263345 22992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:18.263607 22992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:18.266773 22992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead"
format_stamp: "Formatted at 2026-08-11 06:57:18 on dist-test-slave-7vjv"
server_key: "0895baa6a8ad24cfc0301cbc10121052"
server_key_iv: "e30b424c5961bc9fc1172b4db5c1191e"
server_key_version: "encryptionkey@0"
I20260811 06:57:18.267589 22992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead"
format_stamp: "Formatted at 2026-08-11 06:57:18 on dist-test-slave-7vjv"
server_key: "0895baa6a8ad24cfc0301cbc10121052"
server_key_iv: "e30b424c5961bc9fc1172b4db5c1191e"
server_key_version: "encryptionkey@0"
I20260811 06:57:18.273046 22992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 06:57:18.276400 23021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:18.277917 22992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:57:18.278102 22992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead"
format_stamp: "Formatted at 2026-08-11 06:57:18 on dist-test-slave-7vjv"
server_key: "0895baa6a8ad24cfc0301cbc10121052"
server_key_iv: "e30b424c5961bc9fc1172b4db5c1191e"
server_key_version: "encryptionkey@0"
I20260811 06:57:18.278311 22992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:18.304399 22992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:18.309072 22992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:18.309408 22992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:18.310451 22992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:18.311990 22992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:18.312104 22992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:18.312291 22992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:18.312369 22992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:18.350399 22992 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:34857
I20260811 06:57:18.350430 23230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:34857 every 8 connection(s)
I20260811 06:57:18.351982 22992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:57:18.360548 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 22992
I20260811 06:57:18.360785 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:57:18.361433 4256 external_mini_cluster.cc:1428] Setting key 22bf908c82870ee5ea1a36963a383a78
Aug 11 06:57:18 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:57:18.373309 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:18.354697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:48037 (local address 127.4.40.62:41667)
0811 06:57:18.355220 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:18.355224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:18.356309 (+ 1085us) server_negotiation.cc:407] Connection header received
0811 06:57:18.358178 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:18.358183 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:18.358268 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:18.358426 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:18.359989 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.360893 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.362656 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.363176 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.363298 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:18.367201 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:18.367208 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:18.367214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:18.367254 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:18.370028 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:18.370815 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:18.370821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:18.370825 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:18.370948 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:18.371386 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:18.371394 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:18.371397 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:18.371707 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:18.371871 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:18.372673 (+ 802us) server_negotiation.cc:299] Negotiation successful
0811 06:57:18.372910 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":258,"threads_started":1}
I20260811 06:57:18.374287 23235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:18.355301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:41667 (local address 127.4.40.1:48037)
0811 06:57:18.356130 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:18.356203 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:18.357890 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:18.358664 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:18.358673 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:18.359206 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:18.359812 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:18.359830 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.361054 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.361058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:18.362494 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:18.362520 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.363391 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.363400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:18.363660 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:18.364473 (+ 813us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:18.364496 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:18.366970 (+ 2474us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:18.370184 (+ 3214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:57:18.370194 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:57:18.370211 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:57:18.370671 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:57:18.371067 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:57:18.371072 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:57:18.371076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:57:18.371271 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:57:18.371846 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:18.371855 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:18.372267 (+ 412us) client_negotiation.cc:769] Sending connection context
0811 06:57:18.372570 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 06:57:18.373128 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":338,"threads_started":1}
I20260811 06:57:18.375881 23231 heartbeater.cc:347] Connected to a master server at 127.4.40.62:41667
I20260811 06:57:18.376251 23231 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:18.377173 23231 heartbeater.cc:511] Master 127.4.40.62:41667 requested a full tablet report, sending...
I20260811 06:57:18.379475 22895 ts_manager.cc:194] Registered new tserver with Master: cb17dbcb6c8b4c9c9b06eeeeb6023ead (127.4.40.1:34857)
I20260811 06:57:18.381049 22895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:48037
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:57:18.433744 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:41667
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:57:18.800777 23240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:18.801074 23240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:18.801306 23240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:18.808375 23240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:18.808519 23240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:18.808732 23240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:57:18.818261 23240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:41667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:18.819643 23240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:18.821060 23240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:18.834321 23245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:18.834494 23246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:18.834494 23248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:18.835395 23240 server_base.cc:1034] running on GCE node
I20260811 06:57:18.835912 23240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:18.836827 23240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:18.838021 23240 hybrid_clock.cc:648] HybridClock initialized: now 1786431438837977 us; error 33 us; skew 500 ppm
I20260811 06:57:18.838270 23240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:57:18 dist-test-slave-7vjv krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:57:18.841624 23240 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:57:18.843116 23240 webserver.cc:460] Webserver started at http://127.4.40.2:38395/ using document root <none> and password file <none>
I20260811 06:57:18.843760 23240 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:18.843859 23240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:18.844255 23240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:18.847122 23240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "cbfb9645dafa4bdb827828e0590e483e"
format_stamp: "Formatted at 2026-08-11 06:57:18 on dist-test-slave-7vjv"
server_key: "aeec546664fe87b9c664be171e98415e"
server_key_iv: "564a01361fd5f03172524f45187398a5"
server_key_version: "encryptionkey@0"
I20260811 06:57:18.847880 23240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "cbfb9645dafa4bdb827828e0590e483e"
format_stamp: "Formatted at 2026-08-11 06:57:18 on dist-test-slave-7vjv"
server_key: "aeec546664fe87b9c664be171e98415e"
server_key_iv: "564a01361fd5f03172524f45187398a5"
server_key_version: "encryptionkey@0"
I20260811 06:57:18.853133 23240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 06:57:18.856577 23262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:18.857872 23240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:57:18.858049 23240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "cbfb9645dafa4bdb827828e0590e483e"
format_stamp: "Formatted at 2026-08-11 06:57:18 on dist-test-slave-7vjv"
server_key: "aeec546664fe87b9c664be171e98415e"
server_key_iv: "564a01361fd5f03172524f45187398a5"
server_key_version: "encryptionkey@0"
I20260811 06:57:18.858264 23240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:18.875586 23240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:18.879896 23240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:18.880267 23240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:18.881198 23240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:18.882705 23240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:18.882815 23240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:18.882975 23240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:18.883070 23240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:18.917254 23240 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:44517
I20260811 06:57:18.917304 23466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:44517 every 8 connection(s)
I20260811 06:57:18.918582 23240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:57:18.925391 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 23240
I20260811 06:57:18.925606 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:57:18.926156 4256 external_mini_cluster.cc:1428] Setting key 84c67e4c4ed4ad93ec4e943d34b26b74
Aug 11 06:57:18 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431438, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:57:18.940182 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:18.921237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:52801 (local address 127.4.40.62:41667)
0811 06:57:18.921812 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:18.921821 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:18.922792 (+ 971us) server_negotiation.cc:407] Connection header received
0811 06:57:18.924586 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:18.924590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:18.924656 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:18.924753 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:18.926650 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.927462 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.929689 (+ 2227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.930091 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.930192 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:18.934386 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:18.934390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:18.934393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:18.934423 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:18.937100 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:18.937823 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:18.937828 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:18.937837 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:18.937962 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:18.938415 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:18.938419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:18.938421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:18.938689 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:18.938848 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:18.939520 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 06:57:18.939772 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":234,"threads_started":1}
I20260811 06:57:18.941077 23470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:18.921885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:41667 (local address 127.4.40.2:52801)
0811 06:57:18.922608 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:18.922650 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:18.924328 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:18.924946 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:18.924953 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:18.925734 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:18.926482 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:18.926507 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.927624 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.927628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:18.929537 (+ 1909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:18.929554 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:18.930321 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:18.930330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:18.930647 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:18.931403 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:18.931442 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:18.934151 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:18.937246 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:57:18.937255 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:57:18.937270 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:57:18.937689 (+ 419us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:57:18.938081 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:57:18.938086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:57:18.938089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:57:18.938284 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:57:18.938823 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:18.938830 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:18.939156 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 06:57:18.939419 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 06:57:18.939944 (+ 525us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":289,"threads_started":1}
I20260811 06:57:18.942660 23467 heartbeater.cc:347] Connected to a master server at 127.4.40.62:41667
I20260811 06:57:18.942981 23467 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:18.943722 23467 heartbeater.cc:511] Master 127.4.40.62:41667 requested a full tablet report, sending...
I20260811 06:57:18.945317 22895 ts_manager.cc:194] Registered new tserver with Master: cbfb9645dafa4bdb827828e0590e483e (127.4.40.2:44517)
I20260811 06:57:18.946154 22895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:52801
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:57:18.995118 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:41667
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:57:19.333107 23475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:57:19.333405 23475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:57:19.333534 23475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:57:19.340968 23475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:57:19.341101 23475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:57:19.341308 23475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:57:19.351055 23475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:41667
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:57:19.352921 23475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:57:19.354800 23475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:57:19.369395 23487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:19.369417 23493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:57:19.369670 23488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:57:19.370218 23475 server_base.cc:1034] running on GCE node
I20260811 06:57:19.370872 23475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:57:19.371804 23475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:57:19.373046 23475 hybrid_clock.cc:648] HybridClock initialized: now 1786431439372988 us; error 51 us; skew 500 ppm
I20260811 06:57:19.373358 23475 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:57:19 dist-test-slave-7vjv krb5kdc[22801](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:57:19.376703 23475 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:57:19.378048 23475 webserver.cc:460] Webserver started at http://127.4.40.3:43689/ using document root <none> and password file <none>
I20260811 06:57:19.378669 23475 fs_manager.cc:359] Metadata directory not provided
I20260811 06:57:19.378775 23475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:57:19.379032 23475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:57:19.381995 23475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "1a58e8a9caeb461982af82d186e8c620"
format_stamp: "Formatted at 2026-08-11 06:57:19 on dist-test-slave-7vjv"
server_key: "dca90dbc8f81fbacee76cca1ecb62bd3"
server_key_iv: "9aa40ce7676e743a1dca992a74c53e65"
server_key_version: "encryptionkey@0"
I20260811 06:57:19.382882 23475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "1a58e8a9caeb461982af82d186e8c620"
format_stamp: "Formatted at 2026-08-11 06:57:19 on dist-test-slave-7vjv"
server_key: "dca90dbc8f81fbacee76cca1ecb62bd3"
server_key_iv: "9aa40ce7676e743a1dca992a74c53e65"
server_key_version: "encryptionkey@0"
I20260811 06:57:19.384940 23231 heartbeater.cc:503] Master 127.4.40.62:41667 was elected leader, sending a full tablet report...
I20260811 06:57:19.388590 23475 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:57:19.392184 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:19.393596 23475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:57:19.393779 23475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "1a58e8a9caeb461982af82d186e8c620"
format_stamp: "Formatted at 2026-08-11 06:57:19 on dist-test-slave-7vjv"
server_key: "dca90dbc8f81fbacee76cca1ecb62bd3"
server_key_iv: "9aa40ce7676e743a1dca992a74c53e65"
server_key_version: "encryptionkey@0"
I20260811 06:57:19.393980 23475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:57:19.410573 23475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:57:19.415067 23475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:57:19.415359 23475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:57:19.416342 23475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:57:19.417627 23475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:57:19.417732 23475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:19.417887 23475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:57:19.417961 23475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:57:19.452066 23475 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:42117
I20260811 06:57:19.452173 23718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:42117 every 8 connection(s)
I20260811 06:57:19.453428 23475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:57:19.454295 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 23475
I20260811 06:57:19.454535 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:57:19.455128 4256 external_mini_cluster.cc:1428] Setting key f6832796a5abd186c45ce68bc69c01f9
Aug 11 06:57:19 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:57:19.474251 23723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.456180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:49817 (local address 127.4.40.62:41667)
0811 06:57:19.456803 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.456811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.458025 (+ 1214us) server_negotiation.cc:407] Connection header received
0811 06:57:19.460253 (+ 2228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.460258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.460336 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.460444 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:19.462013 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.462958 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.464617 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.465058 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.465167 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.468479 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:19.468484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:19.468488 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:19.468528 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:19.471566 (+ 3038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.472319 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.472323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.472325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.472437 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.472808 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.472811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.472813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.473080 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:19.473186 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.473791 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.473954 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":327,"threads_started":1}
I20260811 06:57:19.475157 23724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.456918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:41667 (local address 127.4.40.3:49817)
0811 06:57:19.457843 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:19.457893 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:19.459800 (+ 1907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:19.460667 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:19.460675 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:19.461215 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:57:19.461840 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.461867 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.463120 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.463124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.464465 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.464478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.465248 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.465252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.465439 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.466109 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:57:19.466131 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:57:19.468276 (+ 2145us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:57:19.471718 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:57:19.471724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:57:19.471735 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:57:19.472212 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:57:19.472528 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:57:19.472531 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:57:19.472534 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:57:19.472711 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:57:19.473191 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:57:19.473196 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:57:19.473481 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 06:57:19.473694 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 06:57:19.474063 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":334,"threads_started":1}
I20260811 06:57:19.476512 23719 heartbeater.cc:347] Connected to a master server at 127.4.40.62:41667
I20260811 06:57:19.476810 23719 heartbeater.cc:464] Registering TS with master...
I20260811 06:57:19.477547 23719 heartbeater.cc:511] Master 127.4.40.62:41667 requested a full tablet report, sending...
I20260811 06:57:19.478909 22895 ts_manager.cc:194] Registered new tserver with Master: 1a58e8a9caeb461982af82d186e8c620 (127.4.40.3:42117)
I20260811 06:57:19.479714 22895 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:49817
I20260811 06:57:19.483558 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:57:19.501189 23723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.488319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.4.40.62:41667)
0811 06:57:19.488481 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.488485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.489011 (+ 526us) server_negotiation.cc:407] Connection header received
0811 06:57:19.489234 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.489238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.489297 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.489403 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:19.490855 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.491594 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.492908 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.493335 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.493471 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.495777 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:19.495782 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:19.495785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:19.495814 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:19.498505 (+ 2691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.499154 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.499161 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.499164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.499289 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.499688 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.499694 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.499697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.500017 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:19.500323 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.500647 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.500873 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:57:19.505025 22895 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32962:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:57:19.512030 22895 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:57:19.537422 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.524427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42117 (local address 127.0.0.1:58704)
0811 06:57:19.525915 (+ 1488us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:19.525983 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:19.526179 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:19.527784 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:19.527791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:19.527823 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:57:19.528309 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.528325 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.530943 (+ 2618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.530950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.533922 (+ 2972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.533942 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.536260 (+ 2318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.536269 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.536523 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.536597 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 06:57:19.536708 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:57:19.536933 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1183,"threads_started":1}
I20260811 06:57:19.538995 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.524794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.4.40.3:42117)
0811 06:57:19.526835 (+ 2041us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.526844 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.526875 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:57:19.526998 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.527006 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.527256 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.528563 (+ 1307us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:57:19.528865 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.530761 (+ 1896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.534175 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.535921 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.538010 (+ 2089us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.538275 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.538355 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.538540 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1891,"thread_start_us":1752,"threads_started":1}
I20260811 06:57:19.546887 23595 tablet_service.cc:1467] Processing CreateTablet for tablet 91eedc540b04472285a04a01edd5d344 (DEFAULT_TABLE table=test-table [id=818970d39df3422d9236f8e4c0141c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:57:19.548434 23747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.527409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44517 (local address 127.0.0.1:33292)
0811 06:57:19.540138 (+ 12729us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:19.540155 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:19.540289 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:19.541290 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:19.541294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:19.541319 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:57:19.541657 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.541669 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.543393 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.543400 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.545649 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.545665 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.547608 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.547615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.547786 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.547824 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:57:19.547922 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 06:57:19.548057 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12592,"thread_start_us":12413,"threads_started":1}
I20260811 06:57:19.548635 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.527917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33292 (local address 127.4.40.2:44517)
0811 06:57:19.539657 (+ 11740us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.539669 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.540424 (+ 755us) server_negotiation.cc:407] Connection header received
0811 06:57:19.540543 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.540552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.540792 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.541023 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:57:19.541870 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.543217 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.545843 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.547388 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.547538 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.547743 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.548002 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.548224 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11540,"thread_start_us":11409,"threads_started":1}
I20260811 06:57:19.549153 23595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91eedc540b04472285a04a01edd5d344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:57:19.550210 23743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.524883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34857 (local address 127.0.0.1:50616)
0811 06:57:19.537697 (+ 12814us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:19.537717 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:19.537827 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:19.542014 (+ 4187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:19.542019 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:19.542047 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:57:19.542271 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.542280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.545921 (+ 3641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.545924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.547026 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.547036 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.549479 (+ 2443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.549485 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.549665 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.549699 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:57:19.549790 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 06:57:19.549929 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12689,"thread_start_us":1042,"threads_started":1}
I20260811 06:57:19.550396 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.525676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.4.40.1:34857)
0811 06:57:19.541398 (+ 15722us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.541408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.541437 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:57:19.541546 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.541556 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.541785 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.542439 (+ 654us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:57:19.542693 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.545782 (+ 3089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.547181 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.549296 (+ 2115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.549446 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.549680 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.549909 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.550065 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15514,"thread_start_us":15354,"threads_started":1}
I20260811 06:57:19.552547 23344 tablet_service.cc:1467] Processing CreateTablet for tablet 91eedc540b04472285a04a01edd5d344 (DEFAULT_TABLE table=test-table [id=818970d39df3422d9236f8e4c0141c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:57:19.553417 23113 tablet_service.cc:1467] Processing CreateTablet for tablet 91eedc540b04472285a04a01edd5d344 (DEFAULT_TABLE table=test-table [id=818970d39df3422d9236f8e4c0141c15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:57:19.553999 23344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91eedc540b04472285a04a01edd5d344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:57:19.554652 23113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91eedc540b04472285a04a01edd5d344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:57:19.567162 23752 tablet_bootstrap.cc:492] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Bootstrap starting.
I20260811 06:57:19.568864 23750 tablet_bootstrap.cc:492] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Bootstrap starting.
I20260811 06:57:19.571045 23753 tablet_bootstrap.cc:492] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Bootstrap starting.
I20260811 06:57:19.571923 23752 tablet_bootstrap.cc:654] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Neither blocks nor log segments found. Creating new log.
I20260811 06:57:19.572695 23750 tablet_bootstrap.cc:654] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Neither blocks nor log segments found. Creating new log.
I20260811 06:57:19.573206 23752 log.cc:824] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Log is configured to *not* fsync() on all Append() calls
I20260811 06:57:19.573889 23750 log.cc:824] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Log is configured to *not* fsync() on all Append() calls
I20260811 06:57:19.574581 23753 tablet_bootstrap.cc:654] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Neither blocks nor log segments found. Creating new log.
I20260811 06:57:19.576022 23753 log.cc:824] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:57:19.576608 23752 tablet_bootstrap.cc:492] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: No bootstrap required, opened a new log
I20260811 06:57:19.576957 23752 ts_tablet_manager.cc:1397] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.003s
I20260811 06:57:19.578833 23750 tablet_bootstrap.cc:492] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: No bootstrap required, opened a new log
I20260811 06:57:19.579100 23750 ts_tablet_manager.cc:1397] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 06:57:19.579753 23753 tablet_bootstrap.cc:492] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: No bootstrap required, opened a new log
I20260811 06:57:19.580003 23753 ts_tablet_manager.cc:1397] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 06:57:19.585554 23752 raft_consensus.cc:348] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.586035 23752 raft_consensus.cc:374] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:57:19.586170 23752 raft_consensus.cc:729] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb17dbcb6c8b4c9c9b06eeeeb6023ead, State: Initialized, Role: FOLLOWER
I20260811 06:57:19.586843 23752 consensus_queue.cc:260] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.588780 23752 ts_tablet_manager.cc:1428] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 06:57:19.589706 23750 raft_consensus.cc:348] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.590143 23750 raft_consensus.cc:374] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:57:19.590251 23750 raft_consensus.cc:729] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a58e8a9caeb461982af82d186e8c620, State: Initialized, Role: FOLLOWER
I20260811 06:57:19.590835 23750 consensus_queue.cc:260] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.591948 23719 heartbeater.cc:503] Master 127.4.40.62:41667 was elected leader, sending a full tablet report...
I20260811 06:57:19.592598 23750 ts_tablet_manager.cc:1428] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Time spent starting tablet: real 0.013s user 0.010s sys 0.005s
I20260811 06:57:19.593106 23753 raft_consensus.cc:348] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.593619 23753 raft_consensus.cc:374] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:57:19.593750 23753 raft_consensus.cc:729] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfb9645dafa4bdb827828e0590e483e, State: Initialized, Role: FOLLOWER
I20260811 06:57:19.594304 23753 consensus_queue.cc:260] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.595925 23467 heartbeater.cc:503] Master 127.4.40.62:41667 was elected leader, sending a full tablet report...
I20260811 06:57:19.596688 23753 ts_tablet_manager.cc:1428] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Time spent starting tablet: real 0.017s user 0.013s sys 0.002s
W20260811 06:57:19.604475 23232 tablet.cc:2369] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:57:19.647303 23763 raft_consensus.cc:482] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:57:19.647650 23763 raft_consensus.cc:504] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.649474 23763 leader_election.cc:288] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a58e8a9caeb461982af82d186e8c620 (127.4.40.3:42117), cbfb9645dafa4bdb827828e0590e483e (127.4.40.2:44517)
I20260811 06:57:19.656265 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.649935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:42769 (local address 127.4.40.3:42117)
0811 06:57:19.650187 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.650196 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.650937 (+ 741us) server_negotiation.cc:407] Connection header received
0811 06:57:19.651139 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.651143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.651247 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.651353 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:57:19.651970 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.652761 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.654515 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.655295 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.655451 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.655571 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.655838 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.655966 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 06:57:19.656339 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.649776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42117 (local address 127.4.40.1:42769)
0811 06:57:19.650788 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:19.650813 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:19.650996 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:19.651386 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:19.651394 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:19.651424 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:57:19.651830 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.651843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.652897 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.652904 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.654346 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.654359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.655508 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.655515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.655676 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.655700 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:57:19.655813 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 06:57:19.655942 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":636,"threads_started":1}
I20260811 06:57:19.656653 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.649909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:54469 (local address 127.4.40.2:44517)
0811 06:57:19.650127 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.650135 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.651224 (+ 1089us) server_negotiation.cc:407] Connection header received
0811 06:57:19.651450 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.651458 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.651544 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.651614 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:57:19.652460 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.653148 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.654843 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.655627 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.655796 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.655898 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.656232 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.656360 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 06:57:19.656849 23772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.649969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44517 (local address 127.4.40.1:54469)
0811 06:57:19.651106 (+ 1137us) negotiation.cc:107] Waiting for socket to connect
0811 06:57:19.651123 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:57:19.651269 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:57:19.651851 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:57:19.651855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:57:19.651883 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:57:19.652278 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.652288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.653302 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.653310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.654680 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:57:19.654693 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.655867 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.655873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:57:19.656031 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.656054 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 06:57:19.656206 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 06:57:19.656326 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"thread_start_us":762,"threads_started":1}
I20260811 06:57:19.657546 23628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91eedc540b04472285a04a01edd5d344" candidate_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a58e8a9caeb461982af82d186e8c620" is_pre_election: true
I20260811 06:57:19.657824 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91eedc540b04472285a04a01edd5d344" candidate_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfb9645dafa4bdb827828e0590e483e" is_pre_election: true
I20260811 06:57:19.658313 23374 raft_consensus.cc:2454] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb17dbcb6c8b4c9c9b06eeeeb6023ead in term 0.
I20260811 06:57:19.658311 23628 raft_consensus.cc:2454] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb17dbcb6c8b4c9c9b06eeeeb6023ead in term 0.
I20260811 06:57:19.659008 23024 leader_election.cc:302] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a58e8a9caeb461982af82d186e8c620, cb17dbcb6c8b4c9c9b06eeeeb6023ead; no voters:
I20260811 06:57:19.659375 23763 raft_consensus.cc:2781] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:57:19.659528 23763 raft_consensus.cc:482] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:57:19.659657 23763 raft_consensus.cc:3037] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:57:19.661952 23763 raft_consensus.cc:504] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
I20260811 06:57:19.662621 23763 leader_election.cc:288] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [CANDIDATE]: Term 1 election: Requested vote from peers 1a58e8a9caeb461982af82d186e8c620 (127.4.40.3:42117), cbfb9645dafa4bdb827828e0590e483e (127.4.40.2:44517)
I20260811 06:57:19.663077 23374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91eedc540b04472285a04a01edd5d344" candidate_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cbfb9645dafa4bdb827828e0590e483e"
I20260811 06:57:19.663072 23628 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91eedc540b04472285a04a01edd5d344" candidate_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a58e8a9caeb461982af82d186e8c620"
I20260811 06:57:19.663275 23374 raft_consensus.cc:3037] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:57:19.663276 23628 raft_consensus.cc:3037] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:57:19.665616 23628 raft_consensus.cc:2454] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb17dbcb6c8b4c9c9b06eeeeb6023ead in term 1.
I20260811 06:57:19.665711 23374 raft_consensus.cc:2454] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb17dbcb6c8b4c9c9b06eeeeb6023ead in term 1.
I20260811 06:57:19.666011 23024 leader_election.cc:302] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a58e8a9caeb461982af82d186e8c620, cb17dbcb6c8b4c9c9b06eeeeb6023ead; no voters:
I20260811 06:57:19.666293 23763 raft_consensus.cc:2781] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:57:19.666779 23763 raft_consensus.cc:686] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 1 LEADER]: Becoming Leader. State: Replica: cb17dbcb6c8b4c9c9b06eeeeb6023ead, State: Running, Role: LEADER
I20260811 06:57:19.667264 23763 consensus_queue.cc:237] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } }
W20260811 06:57:19.670619 23468 tablet.cc:2369] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:57:19.672223 22891 catalog_manager.cc:5495] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead reported cstate change: term changed from 0 to 1, leader changed from <none> to cb17dbcb6c8b4c9c9b06eeeeb6023ead (127.4.40.1). New cstate: current_term: 1 leader_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb17dbcb6c8b4c9c9b06eeeeb6023ead" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 } health_report { overall_health: UNKNOWN } } }
I20260811 06:57:19.699379 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.694457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50628 (local address 127.4.40.1:34857)
0811 06:57:19.694642 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.694647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.694736 (+ 89us) server_negotiation.cc:407] Connection header received
0811 06:57:19.694898 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.694902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.694978 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.695099 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:57:19.695455 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.696256 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.697465 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.697900 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.698005 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.698431 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:57:19.698903 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:57:19.699070 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.699111 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.699176 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 06:57:19.705065 23720 tablet.cc:2369] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:57:19.707386 23374 raft_consensus.cc:1261] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 1 FOLLOWER]: Refusing update from remote peer cb17dbcb6c8b4c9c9b06eeeeb6023ead: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:57:19.707386 23628 raft_consensus.cc:1261] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 1 FOLLOWER]: Refusing update from remote peer cb17dbcb6c8b4c9c9b06eeeeb6023ead: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:57:19.708582 23774 consensus_queue.cc:1035] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a58e8a9caeb461982af82d186e8c620" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:57:19.708945 23763 consensus_queue.cc:1035] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [LEADER]: Connected to new peer: Peer: permanent_uuid: "cbfb9645dafa4bdb827828e0590e483e" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:57:19.720423 23788 mvcc.cc:205] Tried to move back new op lower bound from 7317223177031114752 to 7317223176881102848. Current Snapshot: MvccSnapshot[applied={T|T < 7317223177031114752}]
I20260811 06:57:19.733935 23789 mvcc.cc:205] Tried to move back new op lower bound from 7317223177031114752 to 7317223176881102848. Current Snapshot: MvccSnapshot[applied={T|T < 7317223177031114752}]
I20260811 06:57:19.745963 22891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:57:19.746281 22891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:32962:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:57:19.750247 22891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b6627d46ded949a99797cb449146fb99: Sending DeleteTablet for 3 replicas of tablet 91eedc540b04472285a04a01edd5d344
I20260811 06:57:19.751354 23595 tablet_service.cc:1514] Processing DeleteTablet for tablet 91eedc540b04472285a04a01edd5d344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:57:19 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:58704
I20260811 06:57:19.751361 23113 tablet_service.cc:1514] Processing DeleteTablet for tablet 91eedc540b04472285a04a01edd5d344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:57:19 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:50616
I20260811 06:57:19.752859 23344 tablet_service.cc:1514] Processing DeleteTablet for tablet 91eedc540b04472285a04a01edd5d344 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:57:19 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:33292
I20260811 06:57:19.756117 23809 tablet_replica.cc:330] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: stopping tablet replica
I20260811 06:57:19.756610 23809 raft_consensus.cc:2229] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:57:19.757180 23809 raft_consensus.cc:2258] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:57:19.757838 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 22992
I20260811 06:57:19.758314 23807 tablet_replica.cc:330] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: stopping tablet replica
I20260811 06:57:19.758736 23807 raft_consensus.cc:2229] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:57:19.759495 23807 raft_consensus.cc:2258] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:57:19.759917 23809 ts_tablet_manager.cc:1905] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:57:19.760354 23808 tablet_replica.cc:330] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: stopping tablet replica
I20260811 06:57:19.760792 23808 raft_consensus.cc:2229] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:57:19.761353 23808 raft_consensus.cc:2258] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:57:19.763417 23807 ts_tablet_manager.cc:1905] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:57:19.767769 23808 ts_tablet_manager.cc:1905] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:57:19.768268 23809 ts_tablet_manager.cc:1918] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:57:19.768419 23809 log.cc:1196] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/91eedc540b04472285a04a01edd5d344
I20260811 06:57:19.768946 23809 ts_tablet_manager.cc:1939] T 91eedc540b04472285a04a01edd5d344 P cbfb9645dafa4bdb827828e0590e483e: Deleting consensus metadata
I20260811 06:57:19.770816 22856 catalog_manager.cc:4841] TS cbfb9645dafa4bdb827828e0590e483e (127.4.40.2:44517): tablet 91eedc540b04472285a04a01edd5d344 (table test-table [id=818970d39df3422d9236f8e4c0141c15]) successfully deleted
Aug 11 06:57:19 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:57:19.773705 23807 ts_tablet_manager.cc:1918] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:57:19.773875 23807 log.cc:1196] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/91eedc540b04472285a04a01edd5d344
I20260811 06:57:19.774461 23807 ts_tablet_manager.cc:1939] T 91eedc540b04472285a04a01edd5d344 P cb17dbcb6c8b4c9c9b06eeeeb6023ead: Deleting consensus metadata
I20260811 06:57:19.776518 22854 catalog_manager.cc:4841] TS cb17dbcb6c8b4c9c9b06eeeeb6023ead (127.4.40.1:34857): tablet 91eedc540b04472285a04a01edd5d344 (table test-table [id=818970d39df3422d9236f8e4c0141c15]) successfully deleted
I20260811 06:57:19.777678 23808 ts_tablet_manager.cc:1918] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:57:19.777863 23808 log.cc:1196] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/91eedc540b04472285a04a01edd5d344
I20260811 06:57:19.778405 23808 ts_tablet_manager.cc:1939] T 91eedc540b04472285a04a01edd5d344 P 1a58e8a9caeb461982af82d186e8c620: Deleting consensus metadata
I20260811 06:57:19.780185 22857 catalog_manager.cc:4841] TS 1a58e8a9caeb461982af82d186e8c620 (127.4.40.3:42117): tablet 91eedc540b04472285a04a01edd5d344 (table test-table [id=818970d39df3422d9236f8e4c0141c15]) successfully deleted
I20260811 06:57:19.782765 23745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.759885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50630 (local address 127.4.40.1:34857)
0811 06:57:19.760237 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.760244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.762216 (+ 1972us) server_negotiation.cc:407] Connection header received
0811 06:57:19.762495 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.762501 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.762579 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.762658 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:19.764903 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.766262 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.768255 (+ 1993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.769059 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.769167 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.774997 (+ 5830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:19.775012 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:19.775028 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:19.775097 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:19.779691 (+ 4594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.780554 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.780560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.780562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.780722 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.781190 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.781194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.781196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.781601 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:19.781903 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.782235 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.782410 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232}
I20260811 06:57:19.783564 23223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:19.880223 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 22992
I20260811 06:57:19.908514 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 23240
Aug 11 06:57:19 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:57:19.924160 23749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:19.909360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33304 (local address 127.4.40.2:44517)
0811 06:57:19.909572 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:19.909580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:19.909601 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:57:19.909789 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:19.909798 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:19.909873 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:19.909981 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:19.911181 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.911983 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.913299 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:19.913702 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:19.913835 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:19.918180 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:19.918192 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:19.918204 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:19.918237 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:19.921264 (+ 3027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.921998 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.922006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.922008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.922129 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:19.922518 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:19.922524 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:19.922527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:19.922924 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:19.923244 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:19.923575 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 06:57:19.923790 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:57:19.924830 23457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:20.015763 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 23240
I20260811 06:57:20.042110 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 23475
Aug 11 06:57:20 dist-test-slave-7vjv krb5kdc[22801](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:57:20.057472 23741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:57:20.042912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58716 (local address 127.4.40.3:42117)
0811 06:57:20.043096 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 06:57:20.043100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:57:20.043232 (+ 132us) server_negotiation.cc:407] Connection header received
0811 06:57:20.043419 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:57:20.043423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:57:20.043500 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:57:20.043589 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:57:20.045072 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:20.045790 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:20.047086 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:57:20.047467 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:57:20.047558 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:57:20.051993 (+ 4435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:57:20.052015 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:57:20.052033 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:57:20.052065 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:57:20.055019 (+ 2954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:20.055643 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:20.055650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:20.055652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:20.055789 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:57:20.056087 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:57:20.056135 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:57:20.056141 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:57:20.056560 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:57:20.056771 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 06:57:20.057070 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 06:57:20.057232 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 06:57:20.058173 23711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:20.147240 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 23475
I20260811 06:57:20.172014 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 22823
I20260811 06:57:20.173314 22943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:57:20.282802 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 22823
2026-08-11T06:57:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4658 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:57:20 dist-test-slave-7vjv krb5kdc[23821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:57:20 dist-test-slave-7vjv krb5kdc[23821](info): set up 2 sockets
Aug 11 06:57:20 dist-test-slave-7vjv krb5kdc[23821](info): commencing operation
krb5kdc: starting...
W20260811 06:57:21.668998 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:57:21 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:57:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:57:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/rangeradmin_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/rangeradmin_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/rangerlookup_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/rangerlookup_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
I20260811 06:57:22.075619 4256 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCgIwdj/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/postgres -l logfile start
2026-08-11 06:57:24.593 UTC [23879] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:57:24.593 UTC [23879] LOG: listening on IPv4 address "127.4.40.20", port 40203
2026-08-11 06:57:24.597 UTC [23879] LOG: listening on Unix socket "/tmp/.s.PGSQL.40203"
2026-08-11 06:57:24.622 UTC [23881] LOG: database system was shut down at 2026-08-11 06:57:23 UTC
2026-08-11 06:57:24.626 UTC [23879] LOG: database system is ready to accept connections
I20260811 06:57:25.960807 4256 mini_postgres.cc:96] Postgres bound to 40203
2026-08-11 06:57:25.989 UTC [23893] FATAL: database "slave" does not exist
127.4.40.20:40203 - accepting connections
I20260811 06:57:25.991051 4256 mini_ranger.cc:161] Starting Ranger...
I20260811 06:57:26.027192 4256 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:57:26.521222 4256 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:57:26.521373 4256 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-admin
2026-08-11 06:57:26,866 [I] DB FLAVOR :POSTGRES
2026-08-11 06:57:26,867 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:57:26,867 [I] Checking connection..
2026-08-11 06:57:26,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:57:27,144 [I] Checking connection passed.
2026-08-11 06:57:27,145 [I] --------- Verifying version history table ---------
2026-08-11 06:57:27,145 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:57:27,396 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:57:27,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:57:27,690 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:57:27,691 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:57:27,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:57:27.950 UTC [23998] WARNING: there is no transaction in progress
2026-08-11 06:57:27,960 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:57:27,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:57:28,208 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:57:28,208 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:57:28,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:57:28,494 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:57:28,746 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7vjv.c.gcp-upstream.internal','N') ;"
2026-08-11 06:57:29,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:57:29,271 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:57:29,271 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:57:29,513 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:57:29,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:57:29,764 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:57:30,017 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:57:30,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:57:30.994 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.011 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.028 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.043 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.062 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.083 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.147 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.154 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.168 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.179 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.189 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.203 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.212 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.219 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.226 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31.233 UTC [24186] WARNING: there is no transaction in progress
2026-08-11 06:57:31,863 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:57:31,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:57:32,120 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:57:32,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:57:32,372 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:57:32,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:57:32,635 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:57:32,636 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:57:32,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:57:33,180 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7vjv.c.gcp-upstream.internal';"
2026-08-11 06:57:33,426 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:57:33,427 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:57:33,680 [I] DB_PATCHES have already been applied
I20260811 06:57:33.686465 4256 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCgIwdj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:57:33.686554 4256 mini_ranger.cc:193] Using host: 127.4.40.20
I20260811 06:57:33.710922 4256 mini_ranger.cc:227] Ranger admin URL: http://127.4.40.20:46767
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:57:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:57:35.197345 4256 mini_ranger.cc:160] Time spent starting Ranger: real 9.206s user 0.000s sys 0.072s
Aug 11, 2026 6:57:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:57:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:57:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:57:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:57:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:57:57.386523 4256 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/rangerkms_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/rangerkms_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.40.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.40.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:57:57 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431477, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:57:57.608736 4256 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:57:57.634408 4256 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:57:58.432017 4256 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:57:58.432242 4256 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms
I20260811 06:57:58.432374 4256 mini_ranger_kms.cc:227] Using postgres at 127.4.40.20:40203
2026-08-11 06:57:58,956 [I] DB FLAVOR :POSTGRES
2026-08-11 06:57:58,957 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:57:58,957 [I] Checking connection
2026-08-11 06:57:58,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:57:59,216 [I] connection success
2026-08-11 06:57:59,217 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:57:59,217 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:57:59,217 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:57:59,469 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:57:59,469 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:57:59,469 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:57:59,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:40203/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:57:59,768 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:58:00.230647 4256 mini_ranger_kms.cc:310] Created kms service
I20260811 06:58:00.415346 4256 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:58:00.486781 4256 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:58:00 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431477, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:00.977105 4256 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:58:00.977553 4256 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:58:00.977662 4256 mini_ranger_kms.cc:240] Using host: 127.4.40.20
I20260811 06:58:01.013064 4256 mini_ranger_kms.cc:276] Ranger KMS PID: 24521
I20260811 06:58:01.013285 4256 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.40.20:42591
Aug 11, 2026 6:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:58:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 06:58:03 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786431483, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.40.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 06:58:03 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431483, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.40.20@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:04.069181 4256 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.460s user 0.116s sys 0.454s
I20260811 06:58:04.069620 4256 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:58:04.069852 4256 mini_ranger_kms.cc:401] 127.4.40.20:42591/kms/v1/keys
Aug 11 06:58:04 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431483, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.40.20@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
Aug 11 06:58:06 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:58:06.651819 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:33319
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:33319
--ranger_config_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:42591/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:58:07.009371 24597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:07.009652 24597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:07.009827 24597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:07.017221 24597 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:58:07.017331 24597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:07.017414 24597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:07.017519 24597 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:58:07.017606 24597 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:58:07.027846 24597 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.40.20:42591/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:33319
--ranger_config_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:33319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:07.029155 24597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:07.030543 24597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:07.044595 24608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:07.045238 24612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:07.045433 24606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:07.045660 24597 server_base.cc:1034] running on GCE node
I20260811 06:58:07.046604 24597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:07.048343 24597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:07.049597 24597 hybrid_clock.cc:648] HybridClock initialized: now 1786431487049536 us; error 48 us; skew 500 ppm
I20260811 06:58:07.049996 24597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:07 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431487, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:07.057487 24597 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:58:07.059231 24597 webserver.cc:460] Webserver started at http://127.4.40.62:38397/ using document root <none> and password file <none>
I20260811 06:58:07.059938 24597 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:07.060153 24597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:07.060493 24597 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:07 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431487, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:07.111519 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "a4a6c6249afb40ba9a75585a435132da"
format_stamp: "Formatted at 2026-08-11 06:58:07 on dist-test-slave-7vjv"
server_key: "9cefdc99c4229940761ad76fbacafb80"
server_key_iv: "e545a4846411df3da588ac81ebd67148"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:07.112462 24597 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "a4a6c6249afb40ba9a75585a435132da"
format_stamp: "Formatted at 2026-08-11 06:58:07 on dist-test-slave-7vjv"
server_key: "9cefdc99c4229940761ad76fbacafb80"
server_key_iv: "e545a4846411df3da588ac81ebd67148"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:07.118029 24597 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:58:07.138173 24626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:07.139810 24597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 06:58:07.140041 24597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "a4a6c6249afb40ba9a75585a435132da"
format_stamp: "Formatted at 2026-08-11 06:58:07 on dist-test-slave-7vjv"
server_key: "9cefdc99c4229940761ad76fbacafb80"
server_key_iv: "e545a4846411df3da588ac81ebd67148"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:07.140333 24597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:07.155375 24597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:07.159497 24597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:07.159818 24597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:07.180490 24597 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:33319
I20260811 06:58:07.180510 24711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:33319 every 8 connection(s)
I20260811 06:58:07.181911 24597 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:58:07.185808 24712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:58:07.189141 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 24597
I20260811 06:58:07.189358 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:58:07.190163 4256 external_mini_cluster.cc:1428] Setting key b6c5f6b3ee08b36a5c30fd4590e0d1aa
I20260811 06:58:07.199950 24712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da: Bootstrap starting.
I20260811 06:58:07.205209 24712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da: Neither blocks nor log segments found. Creating new log.
I20260811 06:58:07.206821 24712 log.cc:824] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da: Log is configured to *not* fsync() on all Append() calls
I20260811 06:58:07.210021 24712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da: No bootstrap required, opened a new log
I20260811 06:58:07.219389 24712 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a6c6249afb40ba9a75585a435132da" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33319 } }
I20260811 06:58:07.219919 24712 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:58:07.220052 24712 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4a6c6249afb40ba9a75585a435132da, State: Initialized, Role: FOLLOWER
I20260811 06:58:07.220803 24712 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a6c6249afb40ba9a75585a435132da" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33319 } }
I20260811 06:58:07.221094 24712 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:58:07.221235 24712 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:58:07.221439 24712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 0 FOLLOWER]: Advancing to term 1
Aug 11 06:58:07 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:07.225402 24712 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a6c6249afb40ba9a75585a435132da" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33319 } }
I20260811 06:58:07.226294 24712 leader_election.cc:302] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4a6c6249afb40ba9a75585a435132da; no voters:
I20260811 06:58:07.227174 24712 leader_election.cc:288] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:58:07.227288 24720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:58:07.230474 24712 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:58:07.233539 24716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:07.192210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46770 (local address 127.4.40.62:33319)
0811 06:58:07.206994 (+ 14784us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:07.207010 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:07.207071 (+ 61us) server_negotiation.cc:407] Connection header received
0811 06:58:07.209697 (+ 2626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:07.209725 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:07.210224 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:07.210656 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:07.212315 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:07.214374 (+ 2059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:07.216295 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:07.217376 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:07.217647 (+ 271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:07.222775 (+ 5128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:07.222787 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:07.222809 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:07.222872 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:07.227827 (+ 4955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:07.228777 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:07.228788 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:07.228799 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:07.229017 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:07.229440 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:07.229446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:07.229450 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:07.230285 (+ 835us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:07.230663 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:07.231096 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 06:58:07.231641 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14508,"thread_start_us":14700,"threads_started":1}
I20260811 06:58:07.240801 24720 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [term 1 LEADER]: Becoming Leader. State: Replica: a4a6c6249afb40ba9a75585a435132da, State: Running, Role: LEADER
I20260811 06:58:07.241545 24720 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a6c6249afb40ba9a75585a435132da" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33319 } }
I20260811 06:58:07.244917 24721 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4a6c6249afb40ba9a75585a435132da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a6c6249afb40ba9a75585a435132da" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33319 } } }
I20260811 06:58:07.245416 24721 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [sys.catalog]: This master's current role is: LEADER
I20260811 06:58:07.246023 24722 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4a6c6249afb40ba9a75585a435132da. Latest consensus state: current_term: 1 leader_uuid: "a4a6c6249afb40ba9a75585a435132da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4a6c6249afb40ba9a75585a435132da" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33319 } } }
I20260811 06:58:07.246532 24722 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da [sys.catalog]: This master's current role is: LEADER
I20260811 06:58:07.248732 24736 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:58:07.257351 24736 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:58:07.258471 24712 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:58:07.291143 24736 catalog_manager.cc:1347] Generated new cluster ID: 13bd53508d6d45dab45f56723792aa8c
I20260811 06:58:07.291349 24736 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:58:07.317606 24736 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:58:07.319785 24736 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:58:07.336558 24736 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da: Generated new TSK 0
I20260811 06:58:07.337848 24736 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:58:08.116207 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:42591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33319
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:58:08 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786431488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:58:08.541178 24798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:08.541450 24798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:08.541599 24798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:08.550472 24798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:08.550576 24798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:08.550735 24798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:58:08.565354 24798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.40.20:42591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:08.567094 24798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:08.568959 24798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:08.586773 24816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:08.587675 24817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:08.599920 24820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:08.600400 24798 server_base.cc:1034] running on GCE node
I20260811 06:58:08.601109 24798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:08.602219 24798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:08.603518 24798 hybrid_clock.cc:648] HybridClock initialized: now 1786431488603423 us; error 90 us; skew 500 ppm
I20260811 06:58:08.603852 24798 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:08 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:08.607301 24798 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:58:08.609272 24798 webserver.cc:460] Webserver started at http://127.4.40.1:35843/ using document root <none> and password file <none>
I20260811 06:58:08.610034 24798 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:08.610201 24798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:08.610594 24798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:08 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:08.633642 24798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "9682393352574a2e822972b3a3bec756"
format_stamp: "Formatted at 2026-08-11 06:58:08 on dist-test-slave-7vjv"
server_key: "84cfda754b6fb7a86c1ac501cf2cc5c9"
server_key_iv: "e149fb9e43caa742b3445c65e4adb816"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:08.637028 24798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "9682393352574a2e822972b3a3bec756"
format_stamp: "Formatted at 2026-08-11 06:58:08 on dist-test-slave-7vjv"
server_key: "84cfda754b6fb7a86c1ac501cf2cc5c9"
server_key_iv: "e149fb9e43caa742b3445c65e4adb816"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:08.644340 24798 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 06:58:08.673539 24835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:08.675621 24798 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.007s sys 0.000s
I20260811 06:58:08.675823 24798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "9682393352574a2e822972b3a3bec756"
format_stamp: "Formatted at 2026-08-11 06:58:08 on dist-test-slave-7vjv"
server_key: "84cfda754b6fb7a86c1ac501cf2cc5c9"
server_key_iv: "e149fb9e43caa742b3445c65e4adb816"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:08.676111 24798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:08.720111 24798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:08.728267 24798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:08.728693 24798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:08.740576 24798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:58:08.749221 24798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:58:08.749419 24798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:08.749645 24798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:58:08.749791 24798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:08.846498 24798 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:41187
I20260811 06:58:08.846537 25044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:41187 every 8 connection(s)
I20260811 06:58:08.848493 24798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:58:08.860591 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 24798
I20260811 06:58:08.860836 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:58:08.861656 4256 external_mini_cluster.cc:1428] Setting key aee5f05f61459d824630ef2be506efe3
Aug 11 06:58:08 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
Aug 11 06:58:08 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
I20260811 06:58:08.930179 25054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:08.852848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33319 (local address 127.4.40.1:34917)
0811 06:58:08.889889 (+ 37041us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:08.889975 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:08.892677 (+ 2702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:08.900901 (+ 8224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:08.900915 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:08.901587 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:58:08.902459 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:08.902486 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:08.904067 (+ 1581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:08.904072 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:08.906110 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:08.906127 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:08.915980 (+ 9853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:08.915991 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:08.916471 (+ 480us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:08.917394 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:58:08.917429 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:58:08.920615 (+ 3186us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:58:08.924985 (+ 4370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:08.924994 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:08.925013 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:08.925637 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:08.926190 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:08.926195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:08.926199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:08.926537 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:08.927355 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:58:08.927378 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:58:08.927822 (+ 444us) client_negotiation.cc:769] Sending connection context
0811 06:58:08.928173 (+ 351us) client_negotiation.cc:240] Negotiation successful
0811 06:58:08.928746 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36676,"thread_start_us":37703,"threads_started":1}
I20260811 06:58:08.931577 25056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:08.853580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:34917 (local address 127.4.40.62:33319)
0811 06:58:08.889232 (+ 35652us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:08.889239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:08.900203 (+ 10964us) server_negotiation.cc:407] Connection header received
0811 06:58:08.900294 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:08.900300 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:08.900422 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:08.900549 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:08.902719 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:08.903860 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:08.906278 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:08.906908 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:08.907054 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:08.920948 (+ 13894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:08.920956 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:08.920960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:08.921013 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:08.924852 (+ 3839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:08.925821 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:08.925827 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:08.925831 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:08.925987 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:08.926746 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:08.926752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:08.926756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:08.927136 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:08.927274 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:08.930711 (+ 3437us) server_negotiation.cc:299] Negotiation successful
0811 06:58:08.931189 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35498,"thread_start_us":35369,"threads_started":1}
I20260811 06:58:08.947096 25046 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33319
I20260811 06:58:08.947465 25046 heartbeater.cc:464] Registering TS with master...
I20260811 06:58:08.948416 25046 heartbeater.cc:511] Master 127.4.40.62:33319 requested a full tablet report, sending...
I20260811 06:58:08.950775 24660 ts_manager.cc:194] Registered new tserver with Master: 9682393352574a2e822972b3a3bec756 (127.4.40.1:41187)
I20260811 06:58:08.952301 24660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:34917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
W20260811 06:58:08.970635 4256 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.40.2: real 0.105s user 0.000s sys 0.026s
I20260811 06:58:08.971632 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:42591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33319
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:58:09 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
W20260811 06:58:09.471714 25062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:09.472029 25062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:09.472234 25062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:09.480203 25062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:09.480387 25062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:09.480594 25062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:58:09.491935 25062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.40.20:42591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:09.493660 25062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:09.495534 25062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:09.515017 25077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:09.515909 25078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:09.517129 25083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:09.517606 25062 server_base.cc:1034] running on GCE node
I20260811 06:58:09.518415 25062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:09.519536 25062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:09.520787 25062 hybrid_clock.cc:648] HybridClock initialized: now 1786431489520740 us; error 37 us; skew 500 ppm
I20260811 06:58:09.521160 25062 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:09 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:09.524590 25062 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:58:09.525962 25062 webserver.cc:460] Webserver started at http://127.4.40.2:35303/ using document root <none> and password file <none>
I20260811 06:58:09.526638 25062 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:09.526746 25062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:09.527063 25062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:09 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:09.543134 25062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "70e001505d2c453489ddd9c2180431e4"
format_stamp: "Formatted at 2026-08-11 06:58:09 on dist-test-slave-7vjv"
server_key: "5a75acb3f8ef8ea3503f9b0416951a9e"
server_key_iv: "51fbf9ad84597275ea8000ab463b40aa"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:09.544303 25062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "70e001505d2c453489ddd9c2180431e4"
format_stamp: "Formatted at 2026-08-11 06:58:09 on dist-test-slave-7vjv"
server_key: "5a75acb3f8ef8ea3503f9b0416951a9e"
server_key_iv: "51fbf9ad84597275ea8000ab463b40aa"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:09.551314 25062 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260811 06:58:09.566053 25101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:09.568153 25062 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:58:09.568338 25062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "70e001505d2c453489ddd9c2180431e4"
format_stamp: "Formatted at 2026-08-11 06:58:09 on dist-test-slave-7vjv"
server_key: "5a75acb3f8ef8ea3503f9b0416951a9e"
server_key_iv: "51fbf9ad84597275ea8000ab463b40aa"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:09.568619 25062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:09.599910 25062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:09.604339 25062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:09.604672 25062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:09.605727 25062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:58:09.607290 25062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:58:09.607398 25062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:09.607551 25062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:58:09.607630 25062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:09.646620 25062 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:37089
I20260811 06:58:09.646641 25319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:37089 every 8 connection(s)
I20260811 06:58:09.647970 25062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:58:09.651232 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 25062
I20260811 06:58:09.651428 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:58:09.652024 4256 external_mini_cluster.cc:1428] Setting key 705f8699d2c5a4897a15b12e3cbf30b4
Aug 11 06:58:09 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431489, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:09.673957 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:09.651022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:41567 (local address 127.4.40.62:33319)
0811 06:58:09.654497 (+ 3475us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:09.654516 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:09.654536 (+ 20us) server_negotiation.cc:407] Connection header received
0811 06:58:09.656404 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:09.656413 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:09.656528 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:09.656648 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:09.658708 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:09.659958 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:09.662418 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:09.663138 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:09.663292 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:09.667520 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:09.667529 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:09.667536 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:09.667591 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:09.670614 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:09.671349 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:09.671359 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:09.671361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:09.671481 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:09.671960 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:09.671964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:09.671966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:09.672338 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:09.672495 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:09.673336 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 06:58:09.673583 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3335,"thread_start_us":3216,"threads_started":1}
I20260811 06:58:09.675094 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:09.651899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33319 (local address 127.4.40.2:41567)
0811 06:58:09.653586 (+ 1687us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:09.653662 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:09.656076 (+ 2414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:09.656930 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:09.656942 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:09.657569 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:58:09.658474 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:09.658513 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:09.660173 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:09.660178 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:09.662236 (+ 2058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:09.662251 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:09.663407 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:09.663414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:09.663699 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:09.664570 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:58:09.664597 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:58:09.667248 (+ 2651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:58:09.670776 (+ 3528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:09.670784 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:09.670800 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:09.671217 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:09.671601 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:09.671605 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:09.671609 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:09.671830 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:09.672467 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:58:09.672475 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:58:09.672792 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 06:58:09.673176 (+ 384us) client_negotiation.cc:240] Negotiation successful
0811 06:58:09.673674 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1372,"thread_start_us":1090,"threads_started":1}
I20260811 06:58:09.676810 25320 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33319
I20260811 06:58:09.677141 25320 heartbeater.cc:464] Registering TS with master...
I20260811 06:58:09.677928 25320 heartbeater.cc:511] Master 127.4.40.62:33319 requested a full tablet report, sending...
I20260811 06:58:09.679388 24660 ts_manager.cc:194] Registered new tserver with Master: 70e001505d2c453489ddd9c2180431e4 (127.4.40.2:37089)
I20260811 06:58:09.680495 24660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:41567
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:58:09.731583 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:42591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33319
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:58:09.956177 25046 heartbeater.cc:503] Master 127.4.40.62:33319 was elected leader, sending a full tablet report...
W20260811 06:58:10.110745 25327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:10.111029 25327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:10.111129 25327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:10.118466 25327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:10.118589 25327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:10.118698 25327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:58:10.128695 25327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.40.20:42591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33319
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:10.129853 25327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:10.131137 25327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:10.144551 25342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:10.144811 25338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:10.144641 25337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:10.145797 25327 server_base.cc:1034] running on GCE node
I20260811 06:58:10.146448 25327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:10.147379 25327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:10.148607 25327 hybrid_clock.cc:648] HybridClock initialized: now 1786431490148556 us; error 61 us; skew 500 ppm
I20260811 06:58:10.148919 25327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:10 dist-test-slave-7vjv krb5kdc[23821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:10.152166 25327 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:58:10.153555 25327 webserver.cc:460] Webserver started at http://127.4.40.3:40393/ using document root <none> and password file <none>
I20260811 06:58:10.154289 25327 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:10.154431 25327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:10.154811 25327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:10 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:10.170689 25327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "05eed6c5bfae4954b57156bae614564e"
format_stamp: "Formatted at 2026-08-11 06:58:10 on dist-test-slave-7vjv"
server_key: "e864b33dd9697d77dc52166dd06ff2e0"
server_key_iv: "d4afe18ca3d61eca848b6b81223175d7"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:10.171566 25327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "05eed6c5bfae4954b57156bae614564e"
format_stamp: "Formatted at 2026-08-11 06:58:10 on dist-test-slave-7vjv"
server_key: "e864b33dd9697d77dc52166dd06ff2e0"
server_key_iv: "d4afe18ca3d61eca848b6b81223175d7"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:10.177450 25327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 06:58:10.193825 25357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:10.195370 25327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:58:10.195544 25327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "05eed6c5bfae4954b57156bae614564e"
format_stamp: "Formatted at 2026-08-11 06:58:10 on dist-test-slave-7vjv"
server_key: "e864b33dd9697d77dc52166dd06ff2e0"
server_key_iv: "d4afe18ca3d61eca848b6b81223175d7"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:10.195763 25327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:10.208314 25327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:10.212492 25327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:10.212802 25327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:10.213743 25327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:58:10.215113 25327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:58:10.215188 25327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:10.215368 25327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:58:10.215420 25327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:10.250260 25327 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:42327
I20260811 06:58:10.250314 25546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:42327 every 8 connection(s)
I20260811 06:58:10.251541 25327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:58:10.256716 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 25327
I20260811 06:58:10.256893 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:58:10.257421 4256 external_mini_cluster.cc:1428] Setting key c24e9917f343575df6783c47fa45d8ca
Aug 11 06:58:10 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431490, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:10.271572 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.254133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:57169 (local address 127.4.40.62:33319)
0811 06:58:10.254663 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.254668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.255351 (+ 683us) server_negotiation.cc:407] Connection header received
0811 06:58:10.257700 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.257706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.257797 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.257890 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:10.259847 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.260957 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.262426 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.262864 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.262957 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.266190 (+ 3233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:10.266194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:10.266197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:10.266228 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:10.268885 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.269530 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.269534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.269536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.269653 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.270058 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.270061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.270063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.270310 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:10.270427 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.271036 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.271199 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":296,"threads_started":1}
I20260811 06:58:10.272754 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.254489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33319 (local address 127.4.40.3:57169)
0811 06:58:10.255182 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.255228 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:10.257377 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:10.258149 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:10.258157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:10.258731 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:58:10.259655 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.259681 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.261104 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.261107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.262294 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.262313 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.263051 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.263056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.263254 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.263847 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:58:10.263867 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:58:10.265991 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:58:10.269005 (+ 3014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:10.269010 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:10.269020 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:10.269426 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:10.269744 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:10.269747 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:10.269749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:10.269950 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:10.270422 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:58:10.270427 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:58:10.270716 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 06:58:10.270947 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 06:58:10.271424 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":296,"threads_started":1}
I20260811 06:58:10.274398 25547 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33319
I20260811 06:58:10.274667 25547 heartbeater.cc:464] Registering TS with master...
I20260811 06:58:10.275476 25547 heartbeater.cc:511] Master 127.4.40.62:33319 requested a full tablet report, sending...
I20260811 06:58:10.276887 24660 ts_manager.cc:194] Registered new tserver with Master: 05eed6c5bfae4954b57156bae614564e (127.4.40.3:42327)
I20260811 06:58:10.277704 24660 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:57169
I20260811 06:58:10.285164 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:58:10.302541 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.289717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42362 (local address 127.4.40.62:33319)
0811 06:58:10.289941 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.289948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.290276 (+ 328us) server_negotiation.cc:407] Connection header received
0811 06:58:10.290475 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.290480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.290550 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.290651 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:10.292182 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.292872 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.294400 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.294808 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.294936 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.297463 (+ 2527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:10.297469 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:10.297472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:10.297501 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:10.299831 (+ 2330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.300575 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.300582 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.300586 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.300712 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.301085 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.301091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.301094 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.301364 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:10.301647 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.301995 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.302251 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 06:58:10.306365 24660 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:58:10.313514 24660 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:58:10.346276 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.326090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56776 (local address 127.4.40.2:37089)
0811 06:58:10.327342 (+ 1252us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.327349 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.327371 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:58:10.327476 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.327483 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.327743 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.327938 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:58:10.330470 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.333849 (+ 3379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.340177 (+ 6328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.342047 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.342245 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.342548 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.345536 (+ 2988us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.345788 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":911,"threads_started":1}
I20260811 06:58:10.348042 25572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.325739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:37089 (local address 127.0.0.1:56776)
0811 06:58:10.326775 (+ 1036us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.326827 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:10.327125 (+ 298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:10.328328 (+ 1203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:10.328339 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:10.328388 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:58:10.330173 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.330211 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.334068 (+ 3857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.334077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.339909 (+ 5832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.339931 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.342750 (+ 2819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.342760 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.343043 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.343117 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 06:58:10.344691 (+ 1574us) client_negotiation.cc:240] Negotiation successful
0811 06:58:10.344991 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":660,"threads_started":1}
I20260811 06:58:10.352514 25197 tablet_service.cc:1467] Processing CreateTablet for tablet f819164ea209455c86b6981e28c2ced0 (DEFAULT_TABLE table=test-table [id=9728af3dc70d449da9a4c8abb9e4e7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:58:10.354045 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.327351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42327 (local address 127.0.0.1:44810)
0811 06:58:10.334150 (+ 6799us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.334174 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:10.334352 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:10.347759 (+ 13407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:10.347764 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:10.347792 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:58:10.348288 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.348302 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.350123 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.350129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.351865 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.351877 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.353188 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.353194 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.353425 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.353459 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 06:58:10.353563 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:58:10.353701 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":293,"threads_started":1}
I20260811 06:58:10.354158 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.327731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44810 (local address 127.4.40.3:42327)
0811 06:58:10.347106 (+ 19375us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.347112 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.347165 (+ 53us) server_negotiation.cc:407] Connection header received
0811 06:58:10.347273 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.347281 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.347551 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.347766 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:58:10.348527 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.349961 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.352015 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.352971 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.353121 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.353388 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.353607 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.353776 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19075,"thread_start_us":18916,"threads_started":1}
I20260811 06:58:10.354383 25197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f819164ea209455c86b6981e28c2ced0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:58:10.355811 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.328004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35936)
0811 06:58:10.347948 (+ 19944us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.347971 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:10.348140 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:10.348784 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:10.348788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:10.348815 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:58:10.349144 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.349155 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.351109 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.351114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.353125 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.353142 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.355025 (+ 1883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.355031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.355205 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.355250 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 06:58:10.355364 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 06:58:10.355491 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19809,"thread_start_us":19671,"threads_started":1}
I20260811 06:58:10.355929 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.328438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35936 (local address 127.4.40.1:41187)
0811 06:58:10.344670 (+ 16232us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.344680 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.348158 (+ 3478us) server_negotiation.cc:407] Connection header received
0811 06:58:10.348303 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.348310 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.348548 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.348737 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:58:10.349333 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.350939 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.353355 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.354798 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.354936 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.355158 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.355414 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.355589 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15915,"thread_start_us":15452,"threads_started":1}
I20260811 06:58:10.357637 25441 tablet_service.cc:1467] Processing CreateTablet for tablet f819164ea209455c86b6981e28c2ced0 (DEFAULT_TABLE table=test-table [id=9728af3dc70d449da9a4c8abb9e4e7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:58:10.358999 25441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f819164ea209455c86b6981e28c2ced0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:58:10.359131 24926 tablet_service.cc:1467] Processing CreateTablet for tablet f819164ea209455c86b6981e28c2ced0 (DEFAULT_TABLE table=test-table [id=9728af3dc70d449da9a4c8abb9e4e7ec]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:58:10.360311 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f819164ea209455c86b6981e28c2ced0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:58:10.370549 25584 tablet_bootstrap.cc:492] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Bootstrap starting.
I20260811 06:58:10.372467 25585 tablet_bootstrap.cc:492] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Bootstrap starting.
I20260811 06:58:10.374486 25584 tablet_bootstrap.cc:654] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Neither blocks nor log segments found. Creating new log.
I20260811 06:58:10.375813 25584 log.cc:824] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Log is configured to *not* fsync() on all Append() calls
I20260811 06:58:10.376139 25585 tablet_bootstrap.cc:654] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Neither blocks nor log segments found. Creating new log.
I20260811 06:58:10.377638 25585 log.cc:824] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:58:10.377962 25589 tablet_bootstrap.cc:492] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Bootstrap starting.
I20260811 06:58:10.381549 25585 tablet_bootstrap.cc:492] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: No bootstrap required, opened a new log
I20260811 06:58:10.381632 25584 tablet_bootstrap.cc:492] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: No bootstrap required, opened a new log
I20260811 06:58:10.381718 25589 tablet_bootstrap.cc:654] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Neither blocks nor log segments found. Creating new log.
I20260811 06:58:10.381848 25585 ts_tablet_manager.cc:1397] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:58:10.381980 25584 ts_tablet_manager.cc:1397] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 06:58:10.383037 25589 log.cc:824] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Log is configured to *not* fsync() on all Append() calls
I20260811 06:58:10.386289 25589 tablet_bootstrap.cc:492] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: No bootstrap required, opened a new log
I20260811 06:58:10.386565 25589 ts_tablet_manager.cc:1397] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 06:58:10.390633 25585 raft_consensus.cc:348] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.390991 25585 raft_consensus.cc:374] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:58:10.391094 25585 raft_consensus.cc:729] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05eed6c5bfae4954b57156bae614564e, State: Initialized, Role: FOLLOWER
I20260811 06:58:10.391698 25585 consensus_queue.cc:260] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.392794 25547 heartbeater.cc:503] Master 127.4.40.62:33319 was elected leader, sending a full tablet report...
I20260811 06:58:10.392704 25584 raft_consensus.cc:348] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.393107 25584 raft_consensus.cc:374] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:58:10.393232 25584 raft_consensus.cc:729] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9682393352574a2e822972b3a3bec756, State: Initialized, Role: FOLLOWER
I20260811 06:58:10.393818 25584 consensus_queue.cc:260] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.394852 25585 ts_tablet_manager.cc:1428] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Time spent starting tablet: real 0.013s user 0.007s sys 0.003s
I20260811 06:58:10.396550 25584 ts_tablet_manager.cc:1428] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Time spent starting tablet: real 0.014s user 0.009s sys 0.005s
I20260811 06:58:10.397547 25589 raft_consensus.cc:348] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.397951 25589 raft_consensus.cc:374] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:58:10.398077 25589 raft_consensus.cc:729] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70e001505d2c453489ddd9c2180431e4, State: Initialized, Role: FOLLOWER
I20260811 06:58:10.398597 25589 consensus_queue.cc:260] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.399765 25320 heartbeater.cc:503] Master 127.4.40.62:33319 was elected leader, sending a full tablet report...
I20260811 06:58:10.400420 25589 ts_tablet_manager.cc:1428] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Time spent starting tablet: real 0.014s user 0.004s sys 0.011s
W20260811 06:58:10.401115 25321 tablet.cc:2369] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:58:10.405623 25594 raft_consensus.cc:482] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:58:10.405938 25594 raft_consensus.cc:504] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.407398 25594 leader_election.cc:288] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70e001505d2c453489ddd9c2180431e4 (127.4.40.2:37089), 05eed6c5bfae4954b57156bae614564e (127.4.40.3:42327)
I20260811 06:58:10.414705 25598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.407731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:37089 (local address 127.4.40.1:58429)
0811 06:58:10.408329 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.408346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:10.408497 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:10.408804 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:10.408808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:10.408834 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:58:10.409301 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.409311 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.410612 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.410615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.412662 (+ 2047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.412677 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.414027 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.414031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.414144 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.414168 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 06:58:10.414241 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 06:58:10.414329 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":3,"thread_start_us":269,"threads_started":1}
I20260811 06:58:10.414707 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.407675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:58429 (local address 127.4.40.2:37089)
0811 06:58:10.407864 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.407868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.408429 (+ 561us) server_negotiation.cc:407] Connection header received
0811 06:58:10.408619 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.408624 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.408688 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.408782 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:58:10.409448 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.410481 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.412807 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.413819 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.413960 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.414060 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.414298 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.414452 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:58:10.415454 25599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.408471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:42327 (local address 127.4.40.1:38557)
0811 06:58:10.409429 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.409442 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:10.409549 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:10.409838 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:10.409840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:10.409858 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:58:10.410176 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.410183 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.411244 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.411250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.413121 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:10.413131 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.414719 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.414729 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:10.414912 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.414951 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:58:10.415045 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 06:58:10.415185 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":625,"threads_started":1}
I20260811 06:58:10.415518 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.408575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:38557 (local address 127.4.40.3:42327)
0811 06:58:10.408807 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.408816 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.409518 (+ 702us) server_negotiation.cc:407] Connection header received
0811 06:58:10.409638 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.409642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.409716 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.409818 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:58:10.410322 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.411105 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.413253 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.414457 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.414583 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.414679 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.415113 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.415256 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:58:10.415716 25236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f819164ea209455c86b6981e28c2ced0" candidate_uuid: "9682393352574a2e822972b3a3bec756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e001505d2c453489ddd9c2180431e4" is_pre_election: true
I20260811 06:58:10.416298 25236 raft_consensus.cc:2454] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9682393352574a2e822972b3a3bec756 in term 0.
I20260811 06:58:10.416405 25474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f819164ea209455c86b6981e28c2ced0" candidate_uuid: "9682393352574a2e822972b3a3bec756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05eed6c5bfae4954b57156bae614564e" is_pre_election: true
I20260811 06:58:10.416925 25474 raft_consensus.cc:2454] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9682393352574a2e822972b3a3bec756 in term 0.
I20260811 06:58:10.416949 24836 leader_election.cc:302] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e001505d2c453489ddd9c2180431e4, 9682393352574a2e822972b3a3bec756; no voters:
I20260811 06:58:10.417356 25594 raft_consensus.cc:2781] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:58:10.417521 25594 raft_consensus.cc:482] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:58:10.417654 25594 raft_consensus.cc:3037] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:58:10.419875 25594 raft_consensus.cc:504] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.420533 25594 leader_election.cc:288] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [CANDIDATE]: Term 1 election: Requested vote from peers 70e001505d2c453489ddd9c2180431e4 (127.4.40.2:37089), 05eed6c5bfae4954b57156bae614564e (127.4.40.3:42327)
I20260811 06:58:10.420933 25236 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f819164ea209455c86b6981e28c2ced0" candidate_uuid: "9682393352574a2e822972b3a3bec756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70e001505d2c453489ddd9c2180431e4"
I20260811 06:58:10.421139 25236 raft_consensus.cc:3037] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:58:10.421075 25474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f819164ea209455c86b6981e28c2ced0" candidate_uuid: "9682393352574a2e822972b3a3bec756" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05eed6c5bfae4954b57156bae614564e"
I20260811 06:58:10.421285 25474 raft_consensus.cc:3037] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:58:10.423465 25236 raft_consensus.cc:2454] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9682393352574a2e822972b3a3bec756 in term 1.
I20260811 06:58:10.423520 25474 raft_consensus.cc:2454] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9682393352574a2e822972b3a3bec756 in term 1.
I20260811 06:58:10.423870 24836 leader_election.cc:302] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70e001505d2c453489ddd9c2180431e4, 9682393352574a2e822972b3a3bec756; no voters:
I20260811 06:58:10.424186 25594 raft_consensus.cc:2781] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:58:10.424700 25594 raft_consensus.cc:686] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 1 LEADER]: Becoming Leader. State: Replica: 9682393352574a2e822972b3a3bec756, State: Running, Role: LEADER
I20260811 06:58:10.425257 25594 consensus_queue.cc:237] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } }
I20260811 06:58:10.429770 24660 catalog_manager.cc:5495] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9682393352574a2e822972b3a3bec756 (127.4.40.1). New cstate: current_term: 1 leader_uuid: "9682393352574a2e822972b3a3bec756" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9682393352574a2e822972b3a3bec756" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41187 } health_report { overall_health: HEALTHY } } }
I20260811 06:58:10.466998 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.461341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.4.40.1:41187)
0811 06:58:10.461566 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.461573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.461591 (+ 18us) server_negotiation.cc:407] Connection header received
0811 06:58:10.461790 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.461794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.461868 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.462034 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:58:10.462430 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.463194 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.464794 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.465191 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.465328 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.465899 (+ 571us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:58:10.466406 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:58:10.466599 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.466650 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.466725 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 06:58:10.475399 25236 raft_consensus.cc:1261] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 1 FOLLOWER]: Refusing update from remote peer 9682393352574a2e822972b3a3bec756: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:58:10.475435 25474 raft_consensus.cc:1261] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 1 FOLLOWER]: Refusing update from remote peer 9682393352574a2e822972b3a3bec756: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:58:10.476516 25600 consensus_queue.cc:1035] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70e001505d2c453489ddd9c2180431e4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 37089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:58:10.476892 25594 consensus_queue.cc:1035] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [LEADER]: Connected to new peer: Peer: permanent_uuid: "05eed6c5bfae4954b57156bae614564e" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 42327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:58:10.491457 25606 mvcc.cc:205] Tried to move back new op lower bound from 7317223384976969728 to 7317223384785600512. Current Snapshot: MvccSnapshot[applied={T|T < 7317223384976969728}]
I20260811 06:58:10.491915 25610 mvcc.cc:205] Tried to move back new op lower bound from 7317223384976969728 to 7317223384785600512. Current Snapshot: MvccSnapshot[applied={T|T < 7317223384976969728}]
I20260811 06:58:10.509038 24659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:58:10.509384 24659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:58:10.513008 24659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4a6c6249afb40ba9a75585a435132da: Sending DeleteTablet for 3 replicas of tablet f819164ea209455c86b6981e28c2ced0
I20260811 06:58:10.514218 25197 tablet_service.cc:1514] Processing DeleteTablet for tablet f819164ea209455c86b6981e28c2ced0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:58:10 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56776
I20260811 06:58:10.514369 25441 tablet_service.cc:1514] Processing DeleteTablet for tablet f819164ea209455c86b6981e28c2ced0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:58:10 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:44810
I20260811 06:58:10.515034 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet f819164ea209455c86b6981e28c2ced0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:58:10 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:35936
I20260811 06:58:10.515376 25620 tablet_replica.cc:330] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: stopping tablet replica
I20260811 06:58:10.515913 25620 raft_consensus.cc:2229] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:58:10.516539 25620 raft_consensus.cc:2258] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:58:10.517663 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 24798
I20260811 06:58:10.527347 25620 ts_tablet_manager.cc:1905] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:58:10.533209 25622 tablet_replica.cc:330] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: stopping tablet replica
I20260811 06:58:10.533833 25622 raft_consensus.cc:2229] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:58:10.540359 25622 raft_consensus.cc:2258] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:58:10.542912 25622 ts_tablet_manager.cc:1905] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:58:10.543629 25621 tablet_replica.cc:330] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: stopping tablet replica
I20260811 06:58:10.544165 25621 raft_consensus.cc:2229] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:58:10.545306 25621 raft_consensus.cc:2258] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:58:10.546733 25620 ts_tablet_manager.cc:1918] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:58:10.546939 25620 log.cc:1196] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/f819164ea209455c86b6981e28c2ced0
Aug 11 06:58:10 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:58:10.547565 25620 ts_tablet_manager.cc:1939] T f819164ea209455c86b6981e28c2ced0 P 70e001505d2c453489ddd9c2180431e4: Deleting consensus metadata
I20260811 06:58:10.549149 25621 ts_tablet_manager.cc:1905] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:58:10.549671 24627 catalog_manager.cc:4841] TS 70e001505d2c453489ddd9c2180431e4 (127.4.40.2:37089): tablet f819164ea209455c86b6981e28c2ced0 (table test-table [id=9728af3dc70d449da9a4c8abb9e4e7ec]) successfully deleted
I20260811 06:58:10.556727 25622 ts_tablet_manager.cc:1918] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:58:10.556989 25622 log.cc:1196] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/f819164ea209455c86b6981e28c2ced0
I20260811 06:58:10.557615 25582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.519091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35952 (local address 127.4.40.1:41187)
0811 06:58:10.519323 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.519332 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.531777 (+ 12445us) server_negotiation.cc:407] Connection header received
0811 06:58:10.532324 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.532333 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.532442 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.532531 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:10.534918 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.536740 (+ 1822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.541329 (+ 4589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.542340 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.542476 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.548833 (+ 6357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:10.548849 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:10.548864 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:10.548917 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:10.553193 (+ 4276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.554129 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.554142 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.554146 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.554328 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.554824 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.554829 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.554833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.555330 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:10.555614 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.556867 (+ 1253us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.557146 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:58:10.557837 25621 ts_tablet_manager.cc:1918] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:58:10.557695 25622 ts_tablet_manager.cc:1939] T f819164ea209455c86b6981e28c2ced0 P 9682393352574a2e822972b3a3bec756: Deleting consensus metadata
I20260811 06:58:10.558126 25621 log.cc:1196] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/f819164ea209455c86b6981e28c2ced0
I20260811 06:58:10.558531 25035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:10.558766 25621 ts_tablet_manager.cc:1939] T f819164ea209455c86b6981e28c2ced0 P 05eed6c5bfae4954b57156bae614564e: Deleting consensus metadata
I20260811 06:58:10.560855 24627 catalog_manager.cc:4841] TS 05eed6c5bfae4954b57156bae614564e (127.4.40.3:42327): tablet f819164ea209455c86b6981e28c2ced0 (table test-table [id=9728af3dc70d449da9a4c8abb9e4e7ec]) successfully deleted
I20260811 06:58:10.659118 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 24798
I20260811 06:58:10.688844 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 25062
W20260811 06:58:10.688779 24627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:41187 (error 108)
I20260811 06:58:10.690783 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.690256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35956)
0811 06:58:10.690473 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.690603 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 06:58:10.691066 24627 catalog_manager.cc:4584] TS 9682393352574a2e822972b3a3bec756 (127.4.40.1:41187): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f819164ea209455c86b6981e28c2ced0: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Aug 11 06:58:10 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:58:10.704663 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.689710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56786 (local address 127.4.40.2:37089)
0811 06:58:10.689895 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.689902 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.690047 (+ 145us) server_negotiation.cc:407] Connection header received
0811 06:58:10.690265 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.690275 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.690377 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.690549 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:10.692166 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.692894 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.694144 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.694597 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.694726 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.698932 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:10.698944 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:10.698958 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:10.698988 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:10.702155 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.702832 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.702842 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.702848 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.702984 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.703297 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.703300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.703302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.703662 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:10.703866 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.704168 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.704323 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 06:58:10.705374 25308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:10.744848 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.744286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35958)
0811 06:58:10.744526 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.744644 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 06:58:10.745882 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.745511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35960)
0811 06:58:10.745663 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.745745 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 06:58:10.796882 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 25062
I20260811 06:58:10.809015 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.805896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35974)
0811 06:58:10.808758 (+ 2862us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.808843 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2725}
I20260811 06:58:10.810089 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.809730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35990)
0811 06:58:10.809892 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.809966 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 06:58:10.827589 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 25327
Aug 11 06:58:10 dist-test-slave-7vjv krb5kdc[23821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:58:10.843590 25578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.828411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44816 (local address 127.4.40.3:42327)
0811 06:58:10.828597 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:10.828616 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:10.828693 (+ 77us) server_negotiation.cc:407] Connection header received
0811 06:58:10.828899 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:10.828904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:10.828994 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:10.829103 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:10.830427 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.831274 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.832719 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:10.833126 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:10.833269 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:10.837888 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:10.837904 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:10.837921 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:10.837954 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:10.840719 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.841413 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.841421 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.841424 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.841546 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:10.841941 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:10.841945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:10.841947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:10.842323 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:10.842643 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:10.842961 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 06:58:10.843222 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:58:10.844310 25539 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:10.890924 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.890308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:35996)
0811 06:58:10.890645 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.890746 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
I20260811 06:58:10.892061 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:10.891648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:36008)
0811 06:58:10.891875 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:10.891946 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 06:58:10.934410 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 25327
I20260811 06:58:10.963824 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 24597
I20260811 06:58:10.965143 24705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:11.080024 25580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:11.079393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41187 (local address 127.0.0.1:36022)
0811 06:58:11.079700 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:11.079801 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:41187: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 06:58:11.080016 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 24597
I20260811 06:58:11.118841 4256 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:58:11.465529 4256 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:58:11.465798 4256 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:58:11.821458 4256 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:58:11.821 UTC [23879] LOG: received smart shutdown request
2026-08-11 06:58:11.825 UTC [23879] LOG: background worker "logical replication launcher" (PID 23889) exited with exit code 1
2026-08-11 06:58:11.826 UTC [23883] LOG: shutting down
2026-08-11 06:58:11.989 UTC [23879] LOG: database system is shut down
2026-08-11T06:58:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (51833 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:58:12 dist-test-slave-7vjv krb5kdc[25651](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:58:12 dist-test-slave-7vjv krb5kdc[25651](info): set up 2 sockets
Aug 11 06:58:12 dist-test-slave-7vjv krb5kdc[25651](info): commencing operation
krb5kdc: starting...
W20260811 06:58:13.587729 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.417s user 0.008s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:58:13 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:58:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:58:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/rangeradmin_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/rangeradmin_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/rangerlookup_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/rangerlookup_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
I20260811 06:58:13.990855 4256 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCgIwdj/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/postgres -l logfile start
2026-08-11 06:58:16.454 UTC [25713] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 06:58:16.454 UTC [25713] LOG: listening on IPv4 address "127.4.40.20", port 42607
2026-08-11 06:58:16.457 UTC [25713] LOG: listening on Unix socket "/tmp/.s.PGSQL.42607"
2026-08-11 06:58:16.481 UTC [25723] LOG: database system was shut down at 2026-08-11 06:58:15 UTC
2026-08-11 06:58:16.484 UTC [25713] LOG: database system is ready to accept connections
I20260811 06:58:17.713631 4256 mini_postgres.cc:96] Postgres bound to 42607
2026-08-11 06:58:17.739 UTC [25746] FATAL: database "slave" does not exist
127.4.40.20:42607 - accepting connections
I20260811 06:58:17.740134 4256 mini_ranger.cc:161] Starting Ranger...
I20260811 06:58:17.772732 4256 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 06:58:18.231958 4256 mini_ranger.cc:87] Created ranger Postgres database
I20260811 06:58:18.232117 4256 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-admin
2026-08-11 06:58:18,551 [I] DB FLAVOR :POSTGRES
2026-08-11 06:58:18,552 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:58:18,552 [I] Checking connection..
2026-08-11 06:58:18,552 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 06:58:18,813 [I] Checking connection passed.
2026-08-11 06:58:18,813 [I] --------- Verifying version history table ---------
2026-08-11 06:58:18,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:58:19,049 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:58:19,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:58:19,314 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 06:58:19,315 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 06:58:19,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 06:58:19.577 UTC [25848] WARNING: there is no transaction in progress
2026-08-11 06:58:19,584 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 06:58:19,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 06:58:19,831 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 06:58:19,831 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:58:19,832 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 06:58:20,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 06:58:20,340 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7vjv.c.gcp-upstream.internal','N') ;"
2026-08-11 06:58:20,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:58:20,846 [I] Table x_portal_user does not exist in database ranger
2026-08-11 06:58:20,846 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:58:21,112 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 06:58:21,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:58:21,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:58:21,632 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 06:58:21,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 06:58:22.585 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.601 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.617 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.632 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.647 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.667 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.719 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.725 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.739 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.749 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.759 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.772 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.781 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.787 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.794 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:22.800 UTC [26038] WARNING: there is no transaction in progress
2026-08-11 06:58:23,432 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 06:58:23,432 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 06:58:23,679 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 06:58:23,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 06:58:23,937 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 06:58:23,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 06:58:24,173 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 06:58:24,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 06:58:24,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 06:58:24,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7vjv.c.gcp-upstream.internal';"
2026-08-11 06:58:24,924 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 06:58:24,925 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 06:58:25,174 [I] DB_PATCHES have already been applied
I20260811 06:58:25.180819 4256 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCgIwdj/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 06:58:25.180912 4256 mini_ranger.cc:193] Using host: 127.4.40.20
I20260811 06:58:25.203852 4256 mini_ranger.cc:227] Ranger admin URL: http://127.4.40.20:55261
Aug 11, 2026 6:58:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:58:25 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:58:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 6:58:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 6:58:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 6:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 6:58:26 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 06:58:26.693475 4256 mini_ranger.cc:160] Time spent starting Ranger: real 8.953s user 0.005s sys 0.061s
Aug 11, 2026 6:58:45 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 6:58:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 6:58:45 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 6:58:45 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 6:58:46 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 06:58:47.175104 4256 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.40.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.40.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/rangerkms_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.40.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/rangerkms_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.40.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.40.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.40.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/HTTP_127.4.40.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 06:58:47 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431527, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 06:58:47.395627 4256 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 06:58:47.424758 4256 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 06:58:48.275728 4256 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 06:58:48.275858 4256 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms
I20260811 06:58:48.275910 4256 mini_ranger_kms.cc:227] Using postgres at 127.4.40.20:42607
2026-08-11 06:58:48,678 [I] DB FLAVOR :POSTGRES
2026-08-11 06:58:48,678 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 06:58:48,679 [I] Checking connection
2026-08-11 06:58:48,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 06:58:48,927 [I] connection success
2026-08-11 06:58:48,928 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 06:58:48,928 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 06:58:48,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 06:58:49,172 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 06:58:49,172 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 06:58:49,172 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 06:58:49,173 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.40.20:42607/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 06:58:49,445 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 06:58:49.909266 4256 mini_ranger_kms.cc:310] Created kms service
I20260811 06:58:50.082958 4256 mini_ranger_kms.cc:326] Created kudu user
I20260811 06:58:50.129530 4256 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 06:58:50 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431527, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:50.538926 4256 mini_ranger_kms.cc:384] Added ranger policy
I20260811 06:58:50.539309 4256 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCgIwdj/build/asan/bin/postgresql.jar:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCgIwdj/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 06:58:50.539387 4256 mini_ranger_kms.cc:240] Using host: 127.4.40.20
I20260811 06:58:50.563695 4256 mini_ranger_kms.cc:276] Ranger KMS PID: 26332
I20260811 06:58:50.563835 4256 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.40.20:54937
Aug 11, 2026 6:58:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 6:58:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 6:58:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 6:58:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 6:58:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 6:58:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 06:58:52.042681 4256 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 4.647s user 0.080s sys 0.350s
I20260811 06:58:52.042963 4256 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 06:58:52.043138 4256 mini_ranger_kms.cc:401] 127.4.40.20:54937/kms/v1/keys
Aug 11 06:58:52 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786431532, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.40.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 06:58:52 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431532, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.40.20@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
Aug 11 06:58:53 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431532, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.40.20@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
Aug 11 06:58:55 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:58:55.659453 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45637
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45637
--ranger_config_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:58:56.004745 26413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:56.005033 26413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:56.005204 26413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:56.013003 26413 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:58:56.013118 26413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:56.013221 26413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:56.013334 26413 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:58:56.013414 26413 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:58:56.023481 26413 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.40.20:54937/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45637
--ranger_config_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:56.024819 26413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:56.026228 26413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:56.038683 26428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:56.038641 26426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:56.038764 26425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:56.039315 26413 server_base.cc:1034] running on GCE node
I20260811 06:58:56.040272 26413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:56.041697 26413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:56.042941 26413 hybrid_clock.cc:648] HybridClock initialized: now 1786431536042886 us; error 55 us; skew 500 ppm
I20260811 06:58:56.043331 26413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:56 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431536, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:56.046768 26413 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:58:56.048348 26413 webserver.cc:460] Webserver started at http://127.4.40.62:40421/ using document root <none> and password file <none>
I20260811 06:58:56.049027 26413 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:56.049131 26413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:56.049439 26413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:56 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431536, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:56.103751 26413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "8efc3f1a8c964dd290c07dfd2f0406e1"
format_stamp: "Formatted at 2026-08-11 06:58:56 on dist-test-slave-7vjv"
server_key: "46d1b92de3dc0d823d12f49dd22912f2"
server_key_iv: "8dc906f6543664e735c204b744c12ee1"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:56.104705 26413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "8efc3f1a8c964dd290c07dfd2f0406e1"
format_stamp: "Formatted at 2026-08-11 06:58:56 on dist-test-slave-7vjv"
server_key: "46d1b92de3dc0d823d12f49dd22912f2"
server_key_iv: "8dc906f6543664e735c204b744c12ee1"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:56.110345 26413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 06:58:56.127612 26440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:56.129387 26413 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:58:56.129568 26413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "8efc3f1a8c964dd290c07dfd2f0406e1"
format_stamp: "Formatted at 2026-08-11 06:58:56 on dist-test-slave-7vjv"
server_key: "46d1b92de3dc0d823d12f49dd22912f2"
server_key_iv: "8dc906f6543664e735c204b744c12ee1"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:56.129802 26413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:56.145371 26413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:56.148849 26413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:56.149202 26413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:56.168247 26413 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45637
I20260811 06:58:56.168269 26527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45637 every 8 connection(s)
I20260811 06:58:56.169585 26413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:58:56.173269 26529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:58:56.179832 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 26413
I20260811 06:58:56.180071 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:58:56.181058 4256 external_mini_cluster.cc:1428] Setting key 6cfb9307c9f627a81738deb7f80338d8
I20260811 06:58:56.184808 26529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Bootstrap starting.
I20260811 06:58:56.188414 26529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Neither blocks nor log segments found. Creating new log.
I20260811 06:58:56.189651 26529 log.cc:824] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:58:56.192766 26529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: No bootstrap required, opened a new log
Aug 11 06:58:56 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:56.201206 26529 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:58:56.201610 26529 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:58:56.201731 26529 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8efc3f1a8c964dd290c07dfd2f0406e1, State: Initialized, Role: FOLLOWER
I20260811 06:58:56.202330 26529 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:58:56.202582 26529 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:58:56.202757 26529 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:58:56.202955 26529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:58:56.204804 26533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:56.183231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55266 (local address 127.4.40.62:45637)
0811 06:58:56.183974 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:56.183985 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:56.184023 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:58:56.185797 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:56.185817 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:56.186161 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:56.186453 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:56.188207 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:56.189732 (+ 1525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:56.191363 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:56.191995 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:56.192197 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:56.196928 (+ 4731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:56.196936 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:56.196952 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:56.197001 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:56.200083 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:56.200838 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:56.200848 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:56.200860 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:56.201024 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:56.201400 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:56.201406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:56.201409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:56.202138 (+ 729us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:56.202463 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:56.202965 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 06:58:56.203389 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":555,"thread_start_us":366,"threads_started":1}
I20260811 06:58:56.205909 26529 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:58:56.206418 26529 leader_election.cc:302] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8efc3f1a8c964dd290c07dfd2f0406e1; no voters:
I20260811 06:58:56.207098 26529 leader_election.cc:288] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:58:56.207242 26535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:58:56.209193 26529 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:58:56.212210 26535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8efc3f1a8c964dd290c07dfd2f0406e1, State: Running, Role: LEADER
I20260811 06:58:56.212790 26535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:58:56.215824 26529 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 06:58:56.216169 26536 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } } }
I20260811 06:58:56.216555 26536 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:58:56.216497 26537 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8efc3f1a8c964dd290c07dfd2f0406e1. Latest consensus state: current_term: 1 leader_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } } }
I20260811 06:58:56.216784 26537 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:58:56.223686 26558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:58:56.230140 26558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:58:56.247275 26558 catalog_manager.cc:1347] Generated new cluster ID: 6e45123976ff4347b49523d236ccfc4a
I20260811 06:58:56.247454 26558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:58:56.263643 26558 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:58:56.266001 26558 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:58:56.276217 26558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Generated new TSK 0
I20260811 06:58:56.277297 26558 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:58:56.999804 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45637
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:58:57 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786431537, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 06:58:57.441421 26604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:57.441907 26604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:57.442133 26604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:57.455471 26604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:57.455736 26604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:57.456064 26604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:58:57.473652 26604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.40.20:54937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:57.475999 26604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:57.478104 26604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:57.495550 26623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:57.502185 26625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:57.502638 26604 server_base.cc:1034] running on GCE node
W20260811 06:58:57.502882 26619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:57.503805 26604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:57.510911 26604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:57.512217 26604 hybrid_clock.cc:648] HybridClock initialized: now 1786431537512142 us; error 64 us; skew 500 ppm
I20260811 06:58:57.512666 26604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:57 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431537, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:57.517406 26604 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:58:57.521759 26604 webserver.cc:460] Webserver started at http://127.4.40.1:39501/ using document root <none> and password file <none>
I20260811 06:58:57.522792 26604 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:57.522998 26604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:57.523454 26604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:57 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431537, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:57.550290 26604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "b9210d157e804ca4aef288a9b77d9b5e"
format_stamp: "Formatted at 2026-08-11 06:58:57 on dist-test-slave-7vjv"
server_key: "78aecb2877e795a7e76796398253c256"
server_key_iv: "57f90e440484a635b05a2917d34614d6"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:57.551182 26604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "b9210d157e804ca4aef288a9b77d9b5e"
format_stamp: "Formatted at 2026-08-11 06:58:57 on dist-test-slave-7vjv"
server_key: "78aecb2877e795a7e76796398253c256"
server_key_iv: "57f90e440484a635b05a2917d34614d6"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:57.557185 26604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:58:57.604928 26634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:57.607105 26604 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.013s sys 0.015s
I20260811 06:58:57.607311 26604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "b9210d157e804ca4aef288a9b77d9b5e"
format_stamp: "Formatted at 2026-08-11 06:58:57 on dist-test-slave-7vjv"
server_key: "78aecb2877e795a7e76796398253c256"
server_key_iv: "57f90e440484a635b05a2917d34614d6"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:57.607604 26604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:57.651700 26604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:57.657285 26604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:57.657627 26604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:57.661355 26604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:58:57.663319 26604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:58:57.663435 26604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:57.663655 26604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:58:57.663764 26604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:57.709244 26604 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:41729
I20260811 06:58:57.709262 26835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:41729 every 8 connection(s)
I20260811 06:58:57.710853 26604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:58:57.716972 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 26604
I20260811 06:58:57.717247 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:58:57.718106 4256 external_mini_cluster.cc:1428] Setting key 5284e1025dcdbf8dcd4dbc13a879e87c
Aug 11 06:58:57 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431537, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:57.777470 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:57.720415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45637 (local address 127.4.40.1:52937)
0811 06:58:57.750231 (+ 29816us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:57.750301 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:57.753138 (+ 2837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:57.754466 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:57.754481 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:57.755058 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:58:57.755780 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:57.755798 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:57.757425 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:57.757432 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:57.759194 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:57.759216 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:57.760476 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:57.760486 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:57.760704 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:57.764330 (+ 3626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:58:57.764376 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:58:57.767456 (+ 3080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:58:57.772160 (+ 4704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:57.772172 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:57.772192 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:57.772851 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:57.773319 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:57.773326 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:57.773344 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:57.773616 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:57.774454 (+ 838us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:58:57.774464 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:58:57.774933 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 06:58:57.775261 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 06:58:57.775903 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29502,"thread_start_us":29248,"threads_started":1}
I20260811 06:58:57.778478 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:57.720730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:52937 (local address 127.4.40.62:45637)
0811 06:58:57.749542 (+ 28812us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:57.749552 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:57.752187 (+ 2635us) server_negotiation.cc:407] Connection header received
0811 06:58:57.753593 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:57.753603 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:57.753717 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:57.753804 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:57.755972 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:57.757239 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:57.759373 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:57.760167 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:57.760298 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:57.767725 (+ 7427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:57.767734 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:57.767742 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:57.767800 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:57.771945 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:57.772999 (+ 1054us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:57.773008 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:57.773015 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:57.773170 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:57.773773 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:57.773781 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:57.773787 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:57.774207 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:57.774356 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:57.777830 (+ 3474us) server_negotiation.cc:299] Negotiation successful
0811 06:58:57.778075 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28601,"thread_start_us":31359,"threads_started":1}
I20260811 06:58:57.780643 26836 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45637
I20260811 06:58:57.780994 26836 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
I20260811 06:58:57.781879 26836 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:58:57.784891 26467 ts_manager.cc:194] Registered new tserver with Master: b9210d157e804ca4aef288a9b77d9b5e (127.4.40.1:41729)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
I20260811 06:58:57.786867 26467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:52937
Aug 11 06:58:57 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431537, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
W20260811 06:58:57.841810 4256 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.40.2: real 0.117s user 0.000s sys 0.029s
I20260811 06:58:57.842759 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45637
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:58:58 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431537, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
W20260811 06:58:58.361974 26848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:58.362447 26848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:58.362654 26848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:58.374897 26848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:58.375046 26848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:58.375260 26848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:58:58.391486 26848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.40.20:54937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:58.392863 26848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:58.394248 26848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:58.408176 26884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:58.408248 26881 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:58.408210 26877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:58.409263 26848 server_base.cc:1034] running on GCE node
I20260811 06:58:58.410037 26848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:58.411016 26848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:58.412177 26848 hybrid_clock.cc:648] HybridClock initialized: now 1786431538412131 us; error 44 us; skew 500 ppm
I20260811 06:58:58.412515 26848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:58 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431538, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:58.415674 26848 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:58:58.417140 26848 webserver.cc:460] Webserver started at http://127.4.40.2:32975/ using document root <none> and password file <none>
I20260811 06:58:58.417824 26848 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:58.417977 26848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:58.418318 26848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:58 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431538, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:58.434419 26848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "2df250e0b6324f0f8133987d82819376"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "67f6ab6690dc56818742189ffe1dd653"
server_key_iv: "b6b17339204b686f26b0042127400503"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:58.435307 26848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "2df250e0b6324f0f8133987d82819376"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "67f6ab6690dc56818742189ffe1dd653"
server_key_iv: "b6b17339204b686f26b0042127400503"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:58.440773 26848 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 06:58:58.453889 26896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:58.455271 26848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:58:58.455463 26848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "2df250e0b6324f0f8133987d82819376"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "67f6ab6690dc56818742189ffe1dd653"
server_key_iv: "b6b17339204b686f26b0042127400503"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:58.455663 26848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:58.489871 26848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:58.493741 26848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:58.493990 26848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:58.494892 26848 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:58:58.496237 26848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:58:58.496304 26848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:58.496387 26848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:58:58.496428 26848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:58.531047 26848 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:33757
I20260811 06:58:58.531086 27084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:33757 every 8 connection(s)
I20260811 06:58:58.532411 26848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:58:58.534718 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 26848
I20260811 06:58:58.534912 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:58:58.535434 4256 external_mini_cluster.cc:1428] Setting key 4ddc814cbaf67cabad6832b5d437fc79
Aug 11 06:58:58 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431538, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:58.554137 27087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:58.535436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:50317 (local address 127.4.40.62:45637)
0811 06:58:58.535947 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:58.535954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:58.537268 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 06:58:58.539110 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:58.539116 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:58.539213 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:58.539344 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:58.540872 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:58.541646 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:58.543229 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:58.543651 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:58.543793 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:58.547506 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:58.547514 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:58.547522 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:58.547569 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:58.550800 (+ 3231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:58.551750 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:58.551754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:58.551757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:58.551882 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:58.552428 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:58.552432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:58.552434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:58.552710 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:58.552822 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:58.553709 (+ 887us) server_negotiation.cc:299] Negotiation successful
0811 06:58:58.553874 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":249,"threads_started":1}
I20260811 06:58:58.555779 27088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:58.536060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45637 (local address 127.4.40.2:50317)
0811 06:58:58.537100 (+ 1040us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:58.537156 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:58.538845 (+ 1689us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:58.539607 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:58.539615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:58.540046 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:58:58.540696 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:58.540713 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:58.541811 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:58.541822 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:58.543074 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:58.543091 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:58.543889 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:58.543896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:58.544205 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:58.544905 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:58:58.544927 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:58:58.547264 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:58:58.550951 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:58.550962 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:58.550981 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:58.551622 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:58.551989 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:58.551996 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:58.552010 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:58.552309 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:58.552850 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:58:58.552860 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:58:58.553289 (+ 429us) client_negotiation.cc:769] Sending connection context
0811 06:58:58.553620 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 06:58:58.554233 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":480,"threads_started":1}
I20260811 06:58:58.557276 27085 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45637
I20260811 06:58:58.557592 27085 heartbeater.cc:464] Registering TS with master...
I20260811 06:58:58.558295 27085 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:58:58.559907 26467 ts_manager.cc:194] Registered new tserver with Master: 2df250e0b6324f0f8133987d82819376 (127.4.40.2:33757)
I20260811 06:58:58.560784 26467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:50317
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:58:58.603904 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45637
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:58:58.790555 26836 heartbeater.cc:503] Master 127.4.40.62:45637 was elected leader, sending a full tablet report...
W20260811 06:58:58.947407 27093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:58.947674 27093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:58.947762 27093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:58:58.954753 27093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:58:58.954851 27093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:58:58.954989 27093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:58:58.964708 27093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.40.20:54937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:58:58.965863 27093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:58:58.967177 27093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:58:58.980636 27101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:58.980681 27105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:58:58.980746 27103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:58:58.981616 27093 server_base.cc:1034] running on GCE node
I20260811 06:58:58.982273 27093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:58:58.983217 27093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:58:58.984436 27093 hybrid_clock.cc:648] HybridClock initialized: now 1786431538984404 us; error 45 us; skew 500 ppm
I20260811 06:58:58.984743 27093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:58:58 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431538, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:58:58.987831 27093 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:58:58.989313 27093 webserver.cc:460] Webserver started at http://127.4.40.3:36957/ using document root <none> and password file <none>
I20260811 06:58:58.989998 27093 fs_manager.cc:359] Metadata directory not provided
I20260811 06:58:58.990161 27093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:58:58.990480 27093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 06:58:58 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431538, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:58:59.007174 27093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "681bce69320848008a04806ce753ef6a"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "b329f86fce66739b685cb4233543c4aa"
server_key_iv: "1471c9375ed001ccad6b8361d59fb7c3"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:59.008065 27093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "681bce69320848008a04806ce753ef6a"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "b329f86fce66739b685cb4233543c4aa"
server_key_iv: "1471c9375ed001ccad6b8361d59fb7c3"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:59.014205 27093 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 06:58:59.031164 27118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:59.032796 27093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:58:59.033023 27093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "681bce69320848008a04806ce753ef6a"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "b329f86fce66739b685cb4233543c4aa"
server_key_iv: "1471c9375ed001ccad6b8361d59fb7c3"
server_key_version: "kuduclusterkey@0"
I20260811 06:58:59.033259 27093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:58:59.051234 27093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:58:59.055073 27093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:58:59.055377 27093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:58:59.056388 27093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:58:59.057739 27093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:58:59.057859 27093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:59.058012 27093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:58:59.058084 27093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:58:59.091428 27093 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:40391
I20260811 06:58:59.091470 27323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:40391 every 8 connection(s)
I20260811 06:58:59.092796 27093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:58:59.093374 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 27093
I20260811 06:58:59.093592 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:58:59.094296 4256 external_mini_cluster.cc:1428] Setting key 9903d245e44c59b142769e091f69ee80
Aug 11 06:58:59 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431538, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:58:59.113605 27331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:59.095499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:46435 (local address 127.4.40.62:45637)
0811 06:58:59.096041 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:59.096046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:59.097231 (+ 1185us) server_negotiation.cc:407] Connection header received
0811 06:58:59.099095 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:59.099101 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:59.099203 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:59.099319 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:59.100762 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.101473 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.103045 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.103483 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.103627 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:59.106890 (+ 3263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:59.106896 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:59.106899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:59.106930 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:59.110463 (+ 3533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.111196 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.111201 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.111205 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.111342 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.111815 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.111820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.111823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.112161 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:59.112316 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:59.112980 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 06:58:59.113228 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":274,"threads_started":1}
I20260811 06:58:59.114559 27332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:59.096244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45637 (local address 127.4.40.3:46435)
0811 06:58:59.097086 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 06:58:59.097132 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:58:59.098834 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:58:59.099549 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:58:59.099557 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:58:59.099966 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:58:59.100595 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:59.100613 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.101643 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.101647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:59.102891 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:58:59.102903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.103708 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.103714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:58:59.103936 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:59.104573 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:58:59.104594 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:58:59.106687 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:58:59.110600 (+ 3913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:59.110608 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:59.110623 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:59.111067 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:59.111458 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:58:59.111463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:58:59.111466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:58:59.111667 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:58:59.112291 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:58:59.112298 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:58:59.112612 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 06:58:59.112942 (+ 330us) client_negotiation.cc:240] Negotiation successful
0811 06:58:59.113446 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":393,"threads_started":1}
I20260811 06:58:59.115821 27324 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45637
I20260811 06:58:59.116169 27324 heartbeater.cc:464] Registering TS with master...
I20260811 06:58:59.116883 27324 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:58:59.118230 26467 ts_manager.cc:194] Registered new tserver with Master: 681bce69320848008a04806ce753ef6a (127.4.40.3:40391)
I20260811 06:58:59.119037 26467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:46435
I20260811 06:58:59.122480 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:58:59.124233 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 26604
Aug 11 06:58:59 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:58:59.140858 27334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:59.125172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.4.40.1:41729)
0811 06:58:59.125858 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:59.125864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:59.125887 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:58:59.125948 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:59.125954 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:59.126116 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:59.126240 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:59.127656 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.128739 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.130170 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.130677 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.130769 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:59.135218 (+ 4449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:59.135229 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:59.135241 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:59.135272 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:59.138316 (+ 3044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.138918 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.138924 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.138926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.139070 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.139376 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.139379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.139381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.139733 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:59.139959 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:59.140263 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 06:58:59.140455 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":372,"threads_started":1}
I20260811 06:58:59.142076 26831 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:59.225767 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 26604
I20260811 06:58:59.250612 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 26848
Aug 11 06:58:59 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:58:59.267146 27337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:59.251552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43864 (local address 127.4.40.2:33757)
0811 06:58:59.252161 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:59.252171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:59.252201 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:58:59.252279 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:59.252286 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:59.252492 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:59.252618 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:59.253876 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.254858 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.256277 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.256811 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.256941 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:59.261364 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:59.261376 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:59.261390 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:59.261422 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:59.264533 (+ 3111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.265289 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.265296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.265298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.265423 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.265760 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.265763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.265765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.266096 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:59.266326 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:59.266655 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 06:58:59.266874 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":302,"threads_started":1}
I20260811 06:58:59.268426 27079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:59.352710 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 26848
I20260811 06:58:59.381062 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 27093
Aug 11 06:58:59 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:58:59.398474 27339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:58:59.382110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.4.40.3:40391)
0811 06:58:59.382856 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 06:58:59.382863 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:58:59.382892 (+ 29us) server_negotiation.cc:407] Connection header received
0811 06:58:59.382954 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:58:59.382961 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:58:59.383185 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:58:59.383360 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:58:59.384827 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.385813 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.387278 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:58:59.387858 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:58:59.387998 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:58:59.392400 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:58:59.392413 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:58:59.392430 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:58:59.392468 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:58:59.395724 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.396587 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.396594 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.396597 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.396740 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:58:59.397125 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:58:59.397129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:58:59.397131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:58:59.397495 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:58:59.397746 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 06:58:59.397977 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 06:58:59.398179 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":370,"threads_started":1}
I20260811 06:58:59.399775 27313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:59.483569 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 27093
I20260811 06:58:59.511023 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 26413
I20260811 06:58:59.512392 26520 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:58:59.619351 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 26413
I20260811 06:58:59.657688 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45637
--webserver_interface=127.4.40.62
--webserver_port=40421
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45637
--ranger_config_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:58:59.994082 27344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:58:59.994463 27344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:58:59.994626 27344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:00.001680 27344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:00.001827 27344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:00.001938 27344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:00.002041 27344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:00.002135 27344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:00.012020 27344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45637
--ranger_config_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=40421
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:00.013329 27344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:00.014741 27344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:00.026589 27352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:00.026626 27351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:00.026682 27356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:00.027628 27344 server_base.cc:1034] running on GCE node
I20260811 06:59:00.028462 27344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:00.029736 27344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:00.030931 27344 hybrid_clock.cc:648] HybridClock initialized: now 1786431540030873 us; error 55 us; skew 500 ppm
I20260811 06:59:00.031205 27344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:00 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:00.034296 27344 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:00.035588 27344 webserver.cc:460] Webserver started at http://127.4.40.62:40421/ using document root <none> and password file <none>
I20260811 06:59:00.036221 27344 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:00.036335 27344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:59:00 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:59:00.117200 27344 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:59:00.120378 27367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:00.121817 27344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:59:00.122052 27344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "8efc3f1a8c964dd290c07dfd2f0406e1"
format_stamp: "Formatted at 2026-08-11 06:58:56 on dist-test-slave-7vjv"
server_key: "46d1b92de3dc0d823d12f49dd22912f2"
server_key_iv: "8dc906f6543664e735c204b744c12ee1"
server_key_version: "kuduclusterkey@0"
I20260811 06:59:00.123126 27344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:00.146631 27344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:00.150462 27344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:00.150763 27344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:00.170502 27344 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45637
I20260811 06:59:00.170540 27461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45637 every 8 connection(s)
I20260811 06:59:00.172212 27344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:00.176945 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 27344
I20260811 06:59:00.177896 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:41729
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:45637
--webserver_port=39501
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:59:00.178967 27463 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:59:00.182236 27463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Bootstrap starting.
I20260811 06:59:00.202176 27463 log.cc:824] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:00.211141 27463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:59:00.211665 27463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Bootstrap complete.
I20260811 06:59:00.220831 27463 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:59:00.221721 27463 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8efc3f1a8c964dd290c07dfd2f0406e1, State: Initialized, Role: FOLLOWER
I20260811 06:59:00.222362 27463 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:59:00.222601 27463 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:59:00.222736 27463 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:59:00.222932 27463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:59:00.226823 27463 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:59:00.227355 27463 leader_election.cc:302] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8efc3f1a8c964dd290c07dfd2f0406e1; no voters:
I20260811 06:59:00.228072 27463 leader_election.cc:288] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:59:00.228225 27469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:59:00.228927 27469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [term 2 LEADER]: Becoming Leader. State: Replica: 8efc3f1a8c964dd290c07dfd2f0406e1, State: Running, Role: LEADER
I20260811 06:59:00.229674 27469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } }
I20260811 06:59:00.231578 27469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8efc3f1a8c964dd290c07dfd2f0406e1. Latest consensus state: current_term: 2 leader_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } } }
I20260811 06:59:00.232133 27469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:00.232386 27472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8efc3f1a8c964dd290c07dfd2f0406e1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45637 } } }
I20260811 06:59:00.232713 27472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:00.236739 27463 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:00.240839 27474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:00.246902 27474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:00.248350 27474 catalog_manager.cc:1259] Loaded cluster ID: 6e45123976ff4347b49523d236ccfc4a
I20260811 06:59:00.248514 27474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:59:00.252434 27474 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:59:00.253911 27474 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Loaded TSK: 0
I20260811 06:59:00.255431 27474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:59:00.257948 27463 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 06:59:00.638779 27467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:00.639324 27467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:00.639593 27467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:00.650655 27467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:00.650873 27467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:00.651108 27467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:59:00.664239 27467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:41729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=39501
--tserver_master_addrs=127.4.40.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:00.665917 27467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:00.667315 27467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:00.698621 27512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:00.700208 27510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:00.704408 27515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:00.704768 27467 server_base.cc:1034] running on GCE node
I20260811 06:59:00.705704 27467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:00.706780 27467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:00.708033 27467 hybrid_clock.cc:648] HybridClock initialized: now 1786431540707978 us; error 48 us; skew 500 ppm
I20260811 06:59:00.708381 27467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:00 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:00.712730 27467 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:59:00.720994 27467 webserver.cc:460] Webserver started at http://127.4.40.1:39501/ using document root <none> and password file <none>
I20260811 06:59:00.722012 27467 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:00.722239 27467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:59:00 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:59:00.753906 27467 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.004s
I20260811 06:59:00.757619 27535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:00.759099 27467 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:59:00.759291 27467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "b9210d157e804ca4aef288a9b77d9b5e"
format_stamp: "Formatted at 2026-08-11 06:58:57 on dist-test-slave-7vjv"
server_key: "78aecb2877e795a7e76796398253c256"
server_key_iv: "57f90e440484a635b05a2917d34614d6"
server_key_version: "kuduclusterkey@0"
I20260811 06:59:00.760427 27467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:00.817824 27467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:00.823443 27467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:00.823763 27467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:00.825088 27467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:00.854681 27467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:00.854944 27467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:59:00.855154 27467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:00.855289 27467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:00.972628 27467 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:41729
I20260811 06:59:00.972679 27745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:41729 every 8 connection(s)
I20260811 06:59:00.974388 27467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:59:00.976248 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 27467
I20260811 06:59:00.976966 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:33757
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:45637
--webserver_port=32975
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:59:01 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431540, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:01.018615 27750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:00.985379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:59461 (local address 127.4.40.62:45637)
0811 06:59:00.991235 (+ 5856us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:00.991255 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:00.991305 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:59:00.991756 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:00.991775 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:00.992295 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:00.996690 (+ 4395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:00.997173 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:00.999051 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.001614 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:01.002758 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.002973 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:01.007391 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:01.007404 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:01.007427 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:01.007481 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:01.013007 (+ 5526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:01.013994 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:01.014003 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:01.014011 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:01.014288 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:01.014938 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:01.014944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:01.014948 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:01.015607 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:01.015805 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:01.016822 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0811 06:59:01.017256 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5607,"thread_start_us":5389,"threads_started":1}
I20260811 06:59:01.018625 27749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:00.984257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45637 (local address 127.4.40.1:59461)
0811 06:59:00.985096 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:00.985148 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:00.988044 (+ 2896us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:00.994564 (+ 6520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:00.994578 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:00.995265 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:00.996394 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:00.996422 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:00.999265 (+ 2843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:00.999271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:01.001439 (+ 2168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:01.001466 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:01.003041 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.003051 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:01.003336 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:01.004349 (+ 1013us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:01.004392 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:01.007056 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:01.013193 (+ 6137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:01.013203 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:01.013222 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:01.013805 (+ 583us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:01.014470 (+ 665us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:01.014475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:01.014479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:01.014775 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:01.015793 (+ 1018us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:01.015801 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:01.016238 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 06:59:01.016585 (+ 347us) client_negotiation.cc:240] Negotiation successful
0811 06:59:01.017216 (+ 631us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":1314,"threads_started":1}
I20260811 06:59:01.023206 27746 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45637
I20260811 06:59:01.023533 27746 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:01.024433 27746 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
W20260811 06:59:01.027724 27746 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:45637 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 06:59:01.027943 27746 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:01.028231 27746 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:59:01.029600 27746 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:01.029919 27746 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
W20260811 06:59:01.031013 27746 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 06:59:01.420423 27751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:01.420871 27751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:01.421096 27751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:01.432780 27751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:01.432969 27751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:01.433187 27751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:59:01.448805 27751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:33757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=32975
--tserver_master_addrs=127.4.40.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:01.450577 27751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:01.452399 27751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:01.480881 27787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:01.480894 27788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:01.492344 27751 server_base.cc:1034] running on GCE node
W20260811 06:59:01.493278 27790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:01.494128 27751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:01.495422 27751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:01.496703 27751 hybrid_clock.cc:648] HybridClock initialized: now 1786431541496630 us; error 61 us; skew 500 ppm
I20260811 06:59:01.497079 27751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:01 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:01.501248 27751 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:59:01.504385 27751 webserver.cc:460] Webserver started at http://127.4.40.2:32975/ using document root <none> and password file <none>
I20260811 06:59:01.505401 27751 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:01.505571 27751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:59:01 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
Aug 11 06:59:01 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786431541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:01.541462 27751 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:59:01.550951 27803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:01.552772 27751 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.009s sys 0.000s
I20260811 06:59:01.552958 27751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "2df250e0b6324f0f8133987d82819376"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "67f6ab6690dc56818742189ffe1dd653"
server_key_iv: "b6b17339204b686f26b0042127400503"
server_key_version: "kuduclusterkey@0"
I20260811 06:59:01.554358 27751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:01.591935 27751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:01.597167 27751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:01.597570 27751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:01.598826 27751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:01.600445 27751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:01.600577 27751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:01.600718 27751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:01.600831 27751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:01.650839 27751 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:33757
I20260811 06:59:01.650888 28002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:33757 every 8 connection(s)
I20260811 06:59:01.652468 27751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:59:01.661690 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 27751
I20260811 06:59:01.662550 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.40.20:54937/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:40391
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:45637
--webserver_port=36957
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:46413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:59:01 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:01.687319 28007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:01.656897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45637 (local address 127.4.40.2:56905)
0811 06:59:01.659172 (+ 2275us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:01.659263 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:01.661925 (+ 2662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:01.663111 (+ 1186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:01.663123 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:01.663756 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:01.664684 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:01.664714 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:01.666578 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.666584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:01.668734 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:01.668750 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:01.670091 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.670100 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:01.670395 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:01.673262 (+ 2867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:01.673306 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:01.677934 (+ 4628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:01.682208 (+ 4274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:01.682218 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:01.682236 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:01.682884 (+ 648us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:01.683395 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:01.683400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:01.683404 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:01.683688 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:01.684500 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:01.684509 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:01.684943 (+ 434us) client_negotiation.cc:769] Sending connection context
0811 06:59:01.685283 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 06:59:01.685859 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1948,"thread_start_us":1691,"threads_started":1}
I20260811 06:59:01.688820 28006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:01.656388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:56905 (local address 127.4.40.62:45637)
0811 06:59:01.660351 (+ 3963us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:01.660355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:01.660376 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:59:01.662396 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:01.662402 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:01.662503 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:01.662632 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:01.664991 (+ 2359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:01.666373 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.668934 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:01.669628 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:01.669765 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:01.678262 (+ 8497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:01.678269 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:01.678274 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:01.678319 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:01.682024 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:01.683055 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:01.683061 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:01.683065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:01.683247 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:01.683834 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:01.683838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:01.683841 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:01.684258 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:01.684409 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:01.688212 (+ 3803us) server_negotiation.cc:299] Negotiation successful
0811 06:59:01.688451 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3835,"thread_start_us":3711,"threads_started":1}
I20260811 06:59:01.691182 28003 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45637
I20260811 06:59:01.691533 28003 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:01.692473 28003 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:59:01.694667 27397 ts_manager.cc:194] Registered new tserver with Master: 2df250e0b6324f0f8133987d82819376 (127.4.40.2:33757)
I20260811 06:59:01.697386 27397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:56905
Aug 11 06:59:01 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:59:02.031157 27746 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:02.031595 27746 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:59:02.032857 27397 ts_manager.cc:194] Registered new tserver with Master: b9210d157e804ca4aef288a9b77d9b5e (127.4.40.1:41729)
I20260811 06:59:02.034086 27397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:59461
W20260811 06:59:02.118929 28008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:02.119336 28008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:02.119537 28008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:02.130870 28008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:02.131011 28008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:02.131183 28008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:59:02.146171 28008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:46413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.40.20:54937/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:40391
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=36957
--tserver_master_addrs=127.4.40.62:45637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:02.147758 28008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:02.149448 28008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:02.165673 28018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:02.173549 28019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:02.187693 28023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:02.187881 28008 server_base.cc:1034] running on GCE node
I20260811 06:59:02.188625 28008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:02.189704 28008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:02.190979 28008 hybrid_clock.cc:648] HybridClock initialized: now 1786431542190916 us; error 52 us; skew 500 ppm
I20260811 06:59:02.191381 28008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:02 dist-test-slave-7vjv krb5kdc[25651](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431542, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:02.194690 28008 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:59:02.201526 28008 webserver.cc:460] Webserver started at http://127.4.40.3:36957/ using document root <none> and password file <none>
I20260811 06:59:02.202266 28008 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:02.202432 28008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 06:59:02 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431542, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:59:02.234404 28008 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.003s sys 0.011s
I20260811 06:59:02.238682 28033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:02.240669 28008 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 06:59:02.240870 28008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "681bce69320848008a04806ce753ef6a"
format_stamp: "Formatted at 2026-08-11 06:58:58 on dist-test-slave-7vjv"
server_key: "b329f86fce66739b685cb4233543c4aa"
server_key_iv: "1471c9375ed001ccad6b8361d59fb7c3"
server_key_version: "kuduclusterkey@0"
I20260811 06:59:02.242385 28008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:02.269804 28008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:02.275099 28008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:02.275470 28008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:02.278719 28008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:02.280792 28008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:02.281001 28008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 06:59:02.281172 28008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:02.281301 28008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 06:59:02 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431541, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for HTTP/127.4.40.20@KRBTEST.COM
I20260811 06:59:02.499786 28008 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:40391
I20260811 06:59:02.499857 28240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:40391 every 8 connection(s)
I20260811 06:59:02.502080 28008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:59:02.505503 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 28008
I20260811 06:59:02.537019 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.506595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35392 (local address 127.4.40.62:45637)
0811 06:59:02.516620 (+ 10025us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.516627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.517828 (+ 1201us) server_negotiation.cc:407] Connection header received
0811 06:59:02.517910 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.517915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.518024 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.518147 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:02.520225 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.521360 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.523357 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.523993 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.524153 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.528325 (+ 4172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:02.528335 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:02.528343 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:02.528464 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:02.532765 (+ 4301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.533676 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.533682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.533685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.533841 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.534440 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.534448 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.534453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.534946 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:02.535281 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.536115 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.536405 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9853,"thread_start_us":9736,"threads_started":1}
Aug 11 06:59:02 dist-test-slave-7vjv krb5kdc[25651](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431542, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:02.557018 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.525056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:36705 (local address 127.4.40.62:45637)
0811 06:59:02.537327 (+ 12271us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.537332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.539867 (+ 2535us) server_negotiation.cc:407] Connection header received
0811 06:59:02.542759 (+ 2892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.542765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.542862 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.542978 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:02.544479 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.545171 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.546665 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.547071 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.547201 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.550479 (+ 3278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:02.550484 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:02.550487 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:02.550514 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:02.553962 (+ 3448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.554662 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.554668 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.554671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.554822 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.555244 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.555249 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.555252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.555590 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:02.555735 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.556422 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.556653 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12157,"thread_start_us":15210,"threads_started":1}
I20260811 06:59:02.557721 28255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.525883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45637 (local address 127.4.40.3:36705)
0811 06:59:02.539707 (+ 13824us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:02.539762 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:02.542383 (+ 2621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:02.543225 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:02.543235 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:02.543679 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:02.544332 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.544350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.545335 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.545338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.546516 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.546534 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.547289 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.547296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.547514 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.548311 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:02.548332 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:02.550258 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:02.554098 (+ 3840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:02.554104 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:02.554116 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:02.554527 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:02.554925 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:02.554928 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:02.554930 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:02.555119 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:02.555708 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:02.555713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:02.555984 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 06:59:02.556259 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 06:59:02.556643 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11825,"thread_start_us":12172,"threads_started":1}
I20260811 06:59:02.559007 28244 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45637
I20260811 06:59:02.559326 28244 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:02.560218 28244 heartbeater.cc:511] Master 127.4.40.62:45637 requested a full tablet report, sending...
I20260811 06:59:02.561640 27397 ts_manager.cc:194] Registered new tserver with Master: 681bce69320848008a04806ce753ef6a (127.4.40.3:40391)
I20260811 06:59:02.562430 27397 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:36705
I20260811 06:59:02.565297 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:59:02.583530 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.569384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35398 (local address 127.4.40.62:45637)
0811 06:59:02.569572 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.569578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.570010 (+ 432us) server_negotiation.cc:407] Connection header received
0811 06:59:02.570149 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.570167 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.570240 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.570374 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:02.571888 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.573058 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.574365 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.575012 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.575142 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.577650 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:02.577657 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:02.577662 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:02.577699 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:02.580890 (+ 3191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.581580 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.581586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.581590 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.581764 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.582114 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.582119 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.582122 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.582481 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:02.582742 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.582956 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.583161 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:59:02.588907 27397 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35398:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:59:02.599941 27397 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:59:02.629261 28287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.615446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:41729 (local address 127.0.0.1:58318)
0811 06:59:02.616245 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:02.616301 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:02.616477 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:02.621920 (+ 5443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:02.621932 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:02.621970 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:02.622483 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.622500 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.624530 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.624535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.626363 (+ 1828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.626375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.628078 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.628085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.628362 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.628440 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 06:59:02.628590 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 06:59:02.628817 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":22,"thread_start_us":943,"threads_started":1}
I20260811 06:59:02.629789 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.616153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58318 (local address 127.4.40.1:41729)
0811 06:59:02.620745 (+ 4592us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.620752 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.620776 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:59:02.620860 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.620865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.621048 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.621310 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:02.622745 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.624368 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.626565 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.627784 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.627925 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.629234 (+ 1309us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.629311 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.629484 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4436,"thread_start_us":4267,"threads_started":1}
I20260811 06:59:02.631603 28288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.615784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:33757 (local address 127.0.0.1:59682)
0811 06:59:02.622210 (+ 6426us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:02.622228 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:02.622356 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:02.623670 (+ 1314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:02.623674 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:02.623700 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:02.624129 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.624140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.626504 (+ 2364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.626512 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.628845 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.628861 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.630725 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.630732 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.630906 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.630944 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:59:02.631032 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 06:59:02.631217 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6253,"mutex_wait_us":10,"thread_start_us":6038,"threads_started":1}
I20260811 06:59:02.631631 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.616448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59682 (local address 127.4.40.2:33757)
0811 06:59:02.621800 (+ 5352us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.621806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.622458 (+ 652us) server_negotiation.cc:407] Connection header received
0811 06:59:02.623216 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.623230 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.623485 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.623680 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:02.624716 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.626361 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.629063 (+ 2702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.630489 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.630649 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.630831 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.631124 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.631294 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5224,"thread_start_us":5119,"threads_started":1}
I20260811 06:59:02.632730 28296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.622198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:40391 (local address 127.0.0.1:39298)
0811 06:59:02.622908 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:02.622925 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:02.623037 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:02.624129 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:02.624133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:02.624160 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:02.624577 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.624596 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.626847 (+ 2251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.626853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.629082 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.629096 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.631919 (+ 2823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.631925 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.632159 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.632196 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:59:02.632299 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:59:02.632420 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":431,"threads_started":1}
I20260811 06:59:02.632866 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.622852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.4.40.3:40391)
0811 06:59:02.623560 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.623567 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.623598 (+ 31us) server_negotiation.cc:407] Connection header received
0811 06:59:02.623689 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.623696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.623899 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.624079 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:02.624808 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.626686 (+ 1878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.629258 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.631400 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.631522 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.631783 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.632359 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.632519 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":386,"threads_started":1}
I20260811 06:59:02.633983 27620 tablet_service.cc:1467] Processing CreateTablet for tablet 18d7968ef8e54de3be17b97073a1868c (DEFAULT_TABLE table=test-table [id=1eec6d9460364450aced93227d55e5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:02.635712 27620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d7968ef8e54de3be17b97073a1868c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:02.635396 27882 tablet_service.cc:1467] Processing CreateTablet for tablet 18d7968ef8e54de3be17b97073a1868c (DEFAULT_TABLE table=test-table [id=1eec6d9460364450aced93227d55e5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:02.636512 27882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d7968ef8e54de3be17b97073a1868c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:02.636404 28121 tablet_service.cc:1467] Processing CreateTablet for tablet 18d7968ef8e54de3be17b97073a1868c (DEFAULT_TABLE table=test-table [id=1eec6d9460364450aced93227d55e5bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:02.637755 28121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18d7968ef8e54de3be17b97073a1868c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:02.648824 28301 tablet_bootstrap.cc:492] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Bootstrap starting.
I20260811 06:59:02.652281 28302 tablet_bootstrap.cc:492] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Bootstrap starting.
I20260811 06:59:02.653671 28301 tablet_bootstrap.cc:654] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:02.655093 28301 log.cc:824] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:02.655877 28302 tablet_bootstrap.cc:654] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:02.657325 28302 log.cc:824] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:02.662978 28305 tablet_bootstrap.cc:492] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Bootstrap starting.
I20260811 06:59:02.664224 28302 tablet_bootstrap.cc:492] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: No bootstrap required, opened a new log
I20260811 06:59:02.664543 28302 ts_tablet_manager.cc:1397] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260811 06:59:02.665683 28301 tablet_bootstrap.cc:492] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: No bootstrap required, opened a new log
I20260811 06:59:02.665977 28301 ts_tablet_manager.cc:1397] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260811 06:59:02.666810 28305 tablet_bootstrap.cc:654] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:02.668079 28305 log.cc:824] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:02.672573 28302 raft_consensus.cc:348] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.672955 28302 raft_consensus.cc:374] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:02.673082 28302 raft_consensus.cc:729] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 681bce69320848008a04806ce753ef6a, State: Initialized, Role: FOLLOWER
I20260811 06:59:02.673677 28302 consensus_queue.cc:260] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.674803 28244 heartbeater.cc:503] Master 127.4.40.62:45637 was elected leader, sending a full tablet report...
I20260811 06:59:02.675547 28302 ts_tablet_manager.cc:1428] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260811 06:59:02.677533 28305 tablet_bootstrap.cc:492] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: No bootstrap required, opened a new log
I20260811 06:59:02.677815 28305 ts_tablet_manager.cc:1397] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Time spent bootstrapping tablet: real 0.015s user 0.012s sys 0.000s
I20260811 06:59:02.678148 28301 raft_consensus.cc:348] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.678601 28301 raft_consensus.cc:374] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:02.678706 28301 raft_consensus.cc:729] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2df250e0b6324f0f8133987d82819376, State: Initialized, Role: FOLLOWER
I20260811 06:59:02.679347 28301 consensus_queue.cc:260] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.680472 28003 heartbeater.cc:503] Master 127.4.40.62:45637 was elected leader, sending a full tablet report...
I20260811 06:59:02.681195 28301 ts_tablet_manager.cc:1428] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Time spent starting tablet: real 0.015s user 0.011s sys 0.005s
I20260811 06:59:02.686477 28305 raft_consensus.cc:348] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.686801 28305 raft_consensus.cc:374] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:02.686929 28305 raft_consensus.cc:729] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9210d157e804ca4aef288a9b77d9b5e, State: Initialized, Role: FOLLOWER
I20260811 06:59:02.687474 28305 consensus_queue.cc:260] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.688452 27746 heartbeater.cc:503] Master 127.4.40.62:45637 was elected leader, sending a full tablet report...
I20260811 06:59:02.688921 28305 ts_tablet_manager.cc:1428] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
W20260811 06:59:02.734359 27747 tablet.cc:2369] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:59:02.768154 28251 tablet.cc:2369] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:02.793843 28313 raft_consensus.cc:482] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:59:02.794167 28313 raft_consensus.cc:504] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.795941 28313 leader_election.cc:288] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 681bce69320848008a04806ce753ef6a (127.4.40.3:40391), 2df250e0b6324f0f8133987d82819376 (127.4.40.2:33757)
I20260811 06:59:02.803273 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.796486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:52725 (local address 127.4.40.3:40391)
0811 06:59:02.796901 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.796907 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.797364 (+ 457us) server_negotiation.cc:407] Connection header received
0811 06:59:02.797988 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.797993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.798078 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.798182 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:02.798820 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.799539 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.801422 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.802223 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.802378 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.802491 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.802858 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.802997 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260811 06:59:02.803277 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.796468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:48395 (local address 127.4.40.2:33757)
0811 06:59:02.796737 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.796745 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.797554 (+ 809us) server_negotiation.cc:407] Connection header received
0811 06:59:02.797677 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.797682 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.797780 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.797890 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:02.798591 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.799376 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.801294 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.802187 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.802344 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.802459 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.802858 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.802992 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:59:02.803454 28316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.796633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:33757 (local address 127.4.40.1:48395)
0811 06:59:02.797428 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:02.797443 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:02.797565 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:02.797932 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:02.797938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:02.797974 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:02.798388 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.798402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.799521 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.799526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.801139 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.801151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.802424 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.802430 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.802673 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.802712 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 06:59:02.802823 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 06:59:02.803024 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":80,"thread_start_us":439,"threads_started":1}
I20260811 06:59:02.803980 28314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.796184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:40391 (local address 127.4.40.1:52725)
0811 06:59:02.797200 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:02.797225 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:02.797826 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:02.798209 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:02.798212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:02.798232 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:02.798670 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.798684 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.799713 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.799717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.801263 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:02.801279 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.802461 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.802467 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:02.802675 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.802713 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:59:02.802823 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 06:59:02.803030 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":94,"thread_start_us":612,"threads_started":1}
I20260811 06:59:02.804572 27925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d7968ef8e54de3be17b97073a1868c" candidate_uuid: "b9210d157e804ca4aef288a9b77d9b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df250e0b6324f0f8133987d82819376" is_pre_election: true
I20260811 06:59:02.805048 28163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d7968ef8e54de3be17b97073a1868c" candidate_uuid: "b9210d157e804ca4aef288a9b77d9b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681bce69320848008a04806ce753ef6a" is_pre_election: true
I20260811 06:59:02.805162 27925 raft_consensus.cc:2454] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9210d157e804ca4aef288a9b77d9b5e in term 0.
I20260811 06:59:02.805502 28163 raft_consensus.cc:2454] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9210d157e804ca4aef288a9b77d9b5e in term 0.
I20260811 06:59:02.805855 27538 leader_election.cc:302] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df250e0b6324f0f8133987d82819376, b9210d157e804ca4aef288a9b77d9b5e; no voters:
I20260811 06:59:02.806337 28313 raft_consensus.cc:2781] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:59:02.806497 28313 raft_consensus.cc:482] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:59:02.806622 28313 raft_consensus.cc:3037] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:02.808871 28313 raft_consensus.cc:504] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.809525 28313 leader_election.cc:288] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [CANDIDATE]: Term 1 election: Requested vote from peers 681bce69320848008a04806ce753ef6a (127.4.40.3:40391), 2df250e0b6324f0f8133987d82819376 (127.4.40.2:33757)
I20260811 06:59:02.809885 27925 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d7968ef8e54de3be17b97073a1868c" candidate_uuid: "b9210d157e804ca4aef288a9b77d9b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2df250e0b6324f0f8133987d82819376"
I20260811 06:59:02.809851 28163 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18d7968ef8e54de3be17b97073a1868c" candidate_uuid: "b9210d157e804ca4aef288a9b77d9b5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "681bce69320848008a04806ce753ef6a"
I20260811 06:59:02.810061 28163 raft_consensus.cc:3037] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:02.810092 27925 raft_consensus.cc:3037] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:02.812309 27925 raft_consensus.cc:2454] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9210d157e804ca4aef288a9b77d9b5e in term 1.
I20260811 06:59:02.812585 28163 raft_consensus.cc:2454] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9210d157e804ca4aef288a9b77d9b5e in term 1.
I20260811 06:59:02.812676 27538 leader_election.cc:302] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2df250e0b6324f0f8133987d82819376, b9210d157e804ca4aef288a9b77d9b5e; no voters:
I20260811 06:59:02.813036 28313 raft_consensus.cc:2781] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:02.813575 28313 raft_consensus.cc:686] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 1 LEADER]: Becoming Leader. State: Replica: b9210d157e804ca4aef288a9b77d9b5e, State: Running, Role: LEADER
I20260811 06:59:02.814080 28313 consensus_queue.cc:237] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } }
I20260811 06:59:02.818414 27397 catalog_manager.cc:5495] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e reported cstate change: term changed from 0 to 1, leader changed from <none> to b9210d157e804ca4aef288a9b77d9b5e (127.4.40.1). New cstate: current_term: 1 leader_uuid: "b9210d157e804ca4aef288a9b77d9b5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9210d157e804ca4aef288a9b77d9b5e" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 41729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 } health_report { overall_health: UNKNOWN } } }
I20260811 06:59:02.872517 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.866870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.4.40.1:41729)
0811 06:59:02.867077 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.867085 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.867107 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:59:02.867363 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.867366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.867465 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.867630 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:59:02.868033 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.868892 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.870244 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.870658 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.870797 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.871354 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:59:02.871850 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:59:02.872013 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.872156 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.872235 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:59:02.880779 27925 raft_consensus.cc:1261] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 1 FOLLOWER]: Refusing update from remote peer b9210d157e804ca4aef288a9b77d9b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:02.880779 28163 raft_consensus.cc:1261] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 1 FOLLOWER]: Refusing update from remote peer b9210d157e804ca4aef288a9b77d9b5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:02.881760 28317 consensus_queue.cc:1035] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "681bce69320848008a04806ce753ef6a" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 40391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:02.882151 28313 consensus_queue.cc:1035] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2df250e0b6324f0f8133987d82819376" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 33757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:02.894850 28324 mvcc.cc:205] Tried to move back new op lower bound from 7317223599629963264 to 7317223599370059776. Current Snapshot: MvccSnapshot[applied={T|T < 7317223599629963264}]
I20260811 06:59:02.909704 27396 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:02.910046 27396 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35398:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:02.913918 27396 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8efc3f1a8c964dd290c07dfd2f0406e1: Sending DeleteTablet for 3 replicas of tablet 18d7968ef8e54de3be17b97073a1868c
I20260811 06:59:02.915079 27882 tablet_service.cc:1514] Processing DeleteTablet for tablet 18d7968ef8e54de3be17b97073a1868c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:02 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59682
I20260811 06:59:02.915179 27620 tablet_service.cc:1514] Processing DeleteTablet for tablet 18d7968ef8e54de3be17b97073a1868c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:02 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:58318
I20260811 06:59:02.915380 28121 tablet_service.cc:1514] Processing DeleteTablet for tablet 18d7968ef8e54de3be17b97073a1868c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:02 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:39298
I20260811 06:59:02.921125 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 27467
I20260811 06:59:02.922860 28339 tablet_replica.cc:330] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: stopping tablet replica
I20260811 06:59:02.923303 28339 raft_consensus.cc:2229] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:59:02.924039 28339 raft_consensus.cc:2258] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:02.928524 28339 ts_tablet_manager.cc:1905] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:02.936828 28339 ts_tablet_manager.cc:1918] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:02.937076 28339 log.cc:1196] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/18d7968ef8e54de3be17b97073a1868c
I20260811 06:59:02.937704 28339 ts_tablet_manager.cc:1939] T 18d7968ef8e54de3be17b97073a1868c P b9210d157e804ca4aef288a9b77d9b5e: Deleting consensus metadata
I20260811 06:59:02.939682 27372 catalog_manager.cc:4841] TS b9210d157e804ca4aef288a9b77d9b5e (127.4.40.1:41729): tablet 18d7968ef8e54de3be17b97073a1868c (table test-table [id=1eec6d9460364450aced93227d55e5bc]) successfully deleted
I20260811 06:59:02.943527 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:02.922765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.4.40.1:41729)
0811 06:59:02.922939 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:02.922944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:02.922963 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:59:02.923015 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:02.923020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:02.923089 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:02.923186 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:02.925440 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.926731 (+ 1291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.929030 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:02.929784 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:02.929914 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:02.933579 (+ 3665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:02.933596 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:02.933612 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:02.933671 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:02.938379 (+ 4708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.939304 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.939312 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.939315 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.939510 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:02.940069 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:02.940074 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:02.940077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:02.940683 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:02.941057 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:02.942676 (+ 1619us) server_negotiation.cc:299] Negotiation successful
0811 06:59:02.942986 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 06:59:02.944612 27737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:02.945273 28340 tablet_replica.cc:330] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: stopping tablet replica
I20260811 06:59:02.945847 28340 raft_consensus.cc:2229] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:02.946131 28336 tablet_replica.cc:330] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: stopping tablet replica
I20260811 06:59:02.946483 28340 raft_consensus.cc:2258] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:02.946620 28336 raft_consensus.cc:2229] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:02.947120 28336 raft_consensus.cc:2258] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:02.949574 28336 ts_tablet_manager.cc:1905] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:02.949950 28340 ts_tablet_manager.cc:1905] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:02.955556 28340 ts_tablet_manager.cc:1918] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:02.955719 28340 log.cc:1196] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/18d7968ef8e54de3be17b97073a1868c
I20260811 06:59:02.956262 28340 ts_tablet_manager.cc:1939] T 18d7968ef8e54de3be17b97073a1868c P 681bce69320848008a04806ce753ef6a: Deleting consensus metadata
I20260811 06:59:02.956774 28336 ts_tablet_manager.cc:1918] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:02.956939 28336 log.cc:1196] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/18d7968ef8e54de3be17b97073a1868c
I20260811 06:59:02.957481 28336 ts_tablet_manager.cc:1939] T 18d7968ef8e54de3be17b97073a1868c P 2df250e0b6324f0f8133987d82819376: Deleting consensus metadata
I20260811 06:59:02.957858 27372 catalog_manager.cc:4841] TS 681bce69320848008a04806ce753ef6a (127.4.40.3:40391): tablet 18d7968ef8e54de3be17b97073a1868c (table test-table [id=1eec6d9460364450aced93227d55e5bc]) successfully deleted
I20260811 06:59:02.959115 27371 catalog_manager.cc:4841] TS 2df250e0b6324f0f8133987d82819376 (127.4.40.2:33757): tablet 18d7968ef8e54de3be17b97073a1868c (table test-table [id=1eec6d9460364450aced93227d55e5bc]) successfully deleted
I20260811 06:59:03.045149 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 27467
I20260811 06:59:03.081243 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 27751
I20260811 06:59:03.094238 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:03.081933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.4.40.2:33757)
0811 06:59:03.082097 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:03.082102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:03.082121 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:59:03.082317 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:03.082321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:03.082391 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:03.082474 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:03.083804 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:03.084535 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:03.085764 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:03.086131 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:03.086216 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:03.088801 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:03.088814 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:03.088830 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:03.088858 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:03.091669 (+ 2811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:03.092481 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:03.092488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:03.092490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:03.092628 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:03.092939 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:03.092942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:03.092944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:03.093290 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:03.093506 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:03.093776 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 06:59:03.093949 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 06:59:03.094883 27995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:03.184396 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 27751
I20260811 06:59:03.211638 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 28008
I20260811 06:59:03.224841 28298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:03.212513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.4.40.3:40391)
0811 06:59:03.212710 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:03.212715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:03.212734 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:59:03.212889 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:03.212893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:03.212971 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:03.213062 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:03.214357 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:03.215060 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:03.216344 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:03.216718 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:03.216812 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:03.219478 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:03.219490 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:03.219506 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:03.219533 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:03.222312 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:03.222957 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:03.222964 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:03.222966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:03.223107 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:03.223420 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:03.223423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:03.223425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:03.223751 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:03.223986 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:03.224316 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 06:59:03.224478 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:59:03.225494 28233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:03.313853 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 28008
I20260811 06:59:03.341014 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 27344
I20260811 06:59:03.342284 27452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:03.452409 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 27344
I20260811 06:59:03.491600 4256 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 06:59:03.837568 4256 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 06:59:03.837803 4256 mini_ranger.cc:66] Stopping Ranger...
I20260811 06:59:04.193836 4256 mini_ranger.cc:68] Stopped Ranger
2026-08-11 06:59:04.194 UTC [25713] LOG: received smart shutdown request
2026-08-11 06:59:04.197 UTC [25713] LOG: background worker "logical replication launcher" (PID 25744) exited with exit code 1
2026-08-11 06:59:04.197 UTC [25727] LOG: shutting down
2026-08-11 06:59:04.347 UTC [25713] LOG: database system is shut down
2026-08-11T06:59:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52356 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:59:04 dist-test-slave-7vjv krb5kdc[28366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:59:04 dist-test-slave-7vjv krb5kdc[28366](info): set up 2 sockets
Aug 11 06:59:04 dist-test-slave-7vjv krb5kdc[28366](info): commencing operation
krb5kdc: starting...
W20260811 06:59:05.914515 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.388s user 0.001s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:59:06 dist-test-slave-7vjv krb5kdc[28366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:59:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:59:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:59:06.195827 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36905
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36905,127.4.40.61:41631,127.4.40.60:46839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:06.538583 28386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:06.538861 28386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:06.538947 28386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:06.545933 28386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:06.546026 28386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:06.546072 28386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:06.546154 28386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:06.546216 28386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:06.555310 28386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36905,127.4.40.61:41631,127.4.40.60:46839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:06.556598 28386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:06.557848 28386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:06.569655 28403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:06.569841 28395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:06.569741 28396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:06.570940 28386 server_base.cc:1034] running on GCE node
I20260811 06:59:06.571756 28386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:06.573176 28386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:06.574419 28386 hybrid_clock.cc:648] HybridClock initialized: now 1786431546574374 us; error 40 us; skew 500 ppm
I20260811 06:59:06.574754 28386 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:06 dist-test-slave-7vjv krb5kdc[28366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:06.577859 28386 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:06.579334 28386 webserver.cc:460] Webserver started at http://127.4.40.62:43027/ using document root <none> and password file <none>
I20260811 06:59:06.580052 28386 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:06.580217 28386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:06.580493 28386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:06.583382 28386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "654b4770e4034816ae6f7b2e9a5716db"
format_stamp: "Formatted at 2026-08-11 06:59:06 on dist-test-slave-7vjv"
server_key: "39687a21d5e73a4e75ff12549e71cbbd"
server_key_iv: "2d8480c09c689950a5d0f8fd84d81acd"
server_key_version: "encryptionkey@0"
I20260811 06:59:06.584295 28386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "654b4770e4034816ae6f7b2e9a5716db"
format_stamp: "Formatted at 2026-08-11 06:59:06 on dist-test-slave-7vjv"
server_key: "39687a21d5e73a4e75ff12549e71cbbd"
server_key_iv: "2d8480c09c689950a5d0f8fd84d81acd"
server_key_version: "encryptionkey@0"
I20260811 06:59:06.589283 28386 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:59:06.592760 28411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:06.594035 28386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:59:06.594228 28386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "654b4770e4034816ae6f7b2e9a5716db"
format_stamp: "Formatted at 2026-08-11 06:59:06 on dist-test-slave-7vjv"
server_key: "39687a21d5e73a4e75ff12549e71cbbd"
server_key_iv: "2d8480c09c689950a5d0f8fd84d81acd"
server_key_version: "encryptionkey@0"
I20260811 06:59:06.594411 28386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:06.619191 28386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:06.623180 28386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:06.623481 28386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:06.642129 28386 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:36905
I20260811 06:59:06.642166 28504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:36905 every 8 connection(s)
I20260811 06:59:06.643481 28386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:06.645231 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 28386
I20260811 06:59:06.645475 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:59:06.646354 4256 external_mini_cluster.cc:1428] Setting key 1342500bffcd10645fd5387eb45be197
I20260811 06:59:06.648159 28506 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:06.658134 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:59:06 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.40.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.61@KRBTEST.COM" created.
I20260811 06:59:06.681231 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.661789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36905 (local address 127.0.0.1:34356)
0811 06:59:06.662554 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.662599 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:06.664718 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:06.665517 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:06.665523 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:06.666238 (+ 715us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:06.666563 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:06.666579 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:06.668175 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:06.668180 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:06.669264 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:06.669277 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:06.670229 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:06.670236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:06.670499 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:06.671171 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:06.671194 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:06.673763 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:06.677062 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:06.677075 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:06.677086 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:06.677471 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:06.677879 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:06.677886 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:06.677889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:06.678058 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:06.678856 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:06.678863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:06.679072 (+ 209us) client_negotiation.cc:769] Sending connection context
0811 06:59:06.679318 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 06:59:06.679891 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":148,"thread_start_us":354,"threads_started":1}
I20260811 06:59:06.681272 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.663066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.4.40.62:36905)
0811 06:59:06.663856 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:06.663864 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:06.663923 (+ 59us) server_negotiation.cc:407] Connection header received
0811 06:59:06.665074 (+ 1151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:06.665080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:06.665294 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:06.665513 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:06.666764 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:06.667981 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:06.669414 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:06.669974 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:06.670182 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:06.673992 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:06.674000 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:06.674015 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:06.674049 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:06.676922 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:06.677616 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:06.677622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:06.677625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:06.677769 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:06.678178 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:06.678188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:06.678190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:06.678706 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:06.678918 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:06.679452 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 06:59:06.679774 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":577,"thread_start_us":383,"threads_started":1}
I20260811 06:59:06.684691 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:59:06.685798 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.685405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55000)
0811 06:59:06.685585 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.685691 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:59:06.686087 28422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
I20260811 06:59:06.687579 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.687225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55016)
0811 06:59:06.687394 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.687451 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:59:06.687888 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:41631: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab.
Entry for principal kudu/127.4.40.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab.
I20260811 06:59:06.711741 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:41631
--webserver_interface=127.4.40.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab
--principal=kudu/127.4.40.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36905,127.4.40.61:41631,127.4.40.60:46839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:59:06.737373 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } attempt: 1
I20260811 06:59:06.738916 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.738202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55018)
0811 06:59:06.738565 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.738717 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260811 06:59:06.740137 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.739642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55034)
0811 06:59:06.739845 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.739941 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 06:59:06.740351 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:41631: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
I20260811 06:59:06.808615 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } attempt: 2
I20260811 06:59:06.810010 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.809412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55038)
0811 06:59:06.809690 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.809804 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 06:59:06.811189 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.810737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55042)
0811 06:59:06.810931 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.811026 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 06:59:06.811475 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:41631: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
I20260811 06:59:06.902758 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } attempt: 3
I20260811 06:59:06.904044 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.903507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55056)
0811 06:59:06.903747 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.903863 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 06:59:06.905287 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:06.904843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55062)
0811 06:59:06.905002 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:06.905100 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:59:06.905508 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:41631: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
I20260811 06:59:07.048789 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } attempt: 4
I20260811 06:59:07.050083 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.049446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55070)
0811 06:59:07.049806 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.049915 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260811 06:59:07.051177 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.050775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55072)
0811 06:59:07.050925 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.051026 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:59:07.051379 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:41631: Network error: Client connection negotiation failed: client connection to 127.4.40.61:41631: connect: Connection refused (error 111)
W20260811 06:59:07.056032 28522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:07.056350 28522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:07.056447 28522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:07.063344 28522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:07.063432 28522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:07.063488 28522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:07.063539 28522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:07.063581 28522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:07.073140 28522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36905,127.4.40.61:41631,127.4.40.60:46839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab
--principal=kudu/127.4.40.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:41631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.40.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:07.074389 28522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:07.075655 28522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:07.087884 28528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:07.087845 28542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:07.087922 28530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:07.089011 28522 server_base.cc:1034] running on GCE node
I20260811 06:59:07.089742 28522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:07.091039 28522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:07.092198 28522 hybrid_clock.cc:648] HybridClock initialized: now 1786431547092141 us; error 38 us; skew 500 ppm
I20260811 06:59:07.092557 28522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:07.095589 28522 init.cc:377] Logged in from keytab as kudu/127.4.40.61@KRBTEST.COM (short username kudu)
I20260811 06:59:07.097215 28522 webserver.cc:460] Webserver started at http://127.4.40.61:44823/ using document root <none> and password file <none>
I20260811 06:59:07.097893 28522 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:07.098050 28522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:07.098381 28522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:07.101598 28522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data/instance:
uuid: "24ab3ac21f224bb5970b3d66d896585e"
format_stamp: "Formatted at 2026-08-11 06:59:07 on dist-test-slave-7vjv"
server_key: "02e82d4d1348df132b59db5e0e05aecd"
server_key_iv: "0c31356271ad2b46de09715b8a8c5388"
server_key_version: "encryptionkey@0"
I20260811 06:59:07.102408 28522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal/instance:
uuid: "24ab3ac21f224bb5970b3d66d896585e"
format_stamp: "Formatted at 2026-08-11 06:59:07 on dist-test-slave-7vjv"
server_key: "02e82d4d1348df132b59db5e0e05aecd"
server_key_iv: "0c31356271ad2b46de09715b8a8c5388"
server_key_version: "encryptionkey@0"
I20260811 06:59:07.107472 28522 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 06:59:07.110982 28560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:07.112394 28522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 06:59:07.112583 28522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal
uuid: "24ab3ac21f224bb5970b3d66d896585e"
format_stamp: "Formatted at 2026-08-11 06:59:07 on dist-test-slave-7vjv"
server_key: "02e82d4d1348df132b59db5e0e05aecd"
server_key_iv: "0c31356271ad2b46de09715b8a8c5388"
server_key_version: "encryptionkey@0"
I20260811 06:59:07.112777 28522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:07.131917 28522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:07.136018 28522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:07.136320 28522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:07.155790 28522 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.61:41631
I20260811 06:59:07.155843 28654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.61:41631 every 8 connection(s)
I20260811 06:59:07.157178 28522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
I20260811 06:59:07.159251 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 28522
I20260811 06:59:07.159456 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-1/wal/instance
I20260811 06:59:07.160056 4256 external_mini_cluster.cc:1428] Setting key 28c207673962f5390173f174242f84e7
I20260811 06:59:07.161438 28655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:07.171322 28655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.40.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.60@KRBTEST.COM" created.
I20260811 06:59:07.192487 28517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.174546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.4.40.62:36905)
0811 06:59:07.174800 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.174807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.175949 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 06:59:07.177826 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.177832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.177929 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.178038 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.179466 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.180299 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.181864 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.182281 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.182431 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.186188 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.186193 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.186197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.186229 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.189195 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.189982 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.189989 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.189993 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.190118 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.190545 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.190551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.190554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.190927 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.191105 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.191858 (+ 753us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.192087 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 06:59:07.193519 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.174994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36905 (local address 127.0.0.1:34358)
0811 06:59:07.175767 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.175812 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.177559 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.178271 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.178279 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.178759 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.179297 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.179314 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.180474 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.180478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.181690 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.181703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.182531 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.182537 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.182761 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.183481 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.183504 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.185955 (+ 2451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.189360 (+ 3405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.189368 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.189383 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.189849 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.190233 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.190238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.190241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.190431 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.191065 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.191074 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.191505 (+ 431us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.191778 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.192304 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":350,"threads_started":1}
I20260811 06:59:07.195330 28655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 06:59:07.211553 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.196196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55086)
0811 06:59:07.196414 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.196432 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.196539 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.197733 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.197740 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.198146 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.198282 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.198289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.200288 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.200292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.201240 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.201252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.202486 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.202490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.202658 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.203252 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.203281 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.205805 (+ 2524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.209036 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.209042 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.209046 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.209426 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.209831 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.209835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.209839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.209991 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.210567 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.210577 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.210712 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.210865 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.211139 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":22}
I20260811 06:59:07.211598 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.196414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.4.40.61:41631)
0811 06:59:07.197205 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.197215 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.197250 (+ 35us) server_negotiation.cc:407] Connection header received
0811 06:59:07.197305 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.197315 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.197536 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.197680 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.198436 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.200136 (+ 1700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.201374 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.202326 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.202465 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.206018 (+ 3553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.206029 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.206043 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.206074 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.208887 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.209565 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.209573 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.209576 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.209707 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.210094 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.210101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.210104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.210443 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.210668 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.211031 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.211234 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":592,"thread_start_us":389,"threads_started":1}
I20260811 06:59:07.214118 28655 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:59:07.215242 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.214815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50074)
0811 06:59:07.214995 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.215098 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 06:59:07.215631 28566 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.217482 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.217076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50088)
0811 06:59:07.217229 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.217358 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 06:59:07.217810 28655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab.
Entry for principal kudu/127.4.40.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab.
I20260811 06:59:07.227990 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:46839
--webserver_interface=127.4.40.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab
--principal=kudu/127.4.40.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36905,127.4.40.61:41631,127.4.40.60:46839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:59:07.267246 28655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 1
I20260811 06:59:07.268543 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.268029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50102)
0811 06:59:07.268274 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.268378 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 06:59:07.269585 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.269262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50116)
0811 06:59:07.269416 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.269480 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 06:59:07.269826 28655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.338140 28655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 2
I20260811 06:59:07.339329 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.338850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50118)
0811 06:59:07.339054 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.339169 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 06:59:07.340394 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.340017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50124)
0811 06:59:07.340204 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.340276 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 06:59:07.340657 28655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.350646 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } attempt: 5
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 06:59:07.364406 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.351338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55088)
0811 06:59:07.351553 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.351576 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.351727 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.352124 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.352131 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.352544 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.352736 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.352745 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.353751 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.353763 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.354602 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.354631 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.355387 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.355392 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.355608 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.356268 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.356287 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.358281 (+ 1994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.362029 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.362033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.362036 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.362405 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.362848 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.362851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.362853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.362976 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.363676 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.363680 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.363805 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.363911 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.364158 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 06:59:07.364965 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.351387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.4.40.61:41631)
0811 06:59:07.351583 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.351589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.351657 (+ 68us) server_negotiation.cc:407] Connection header received
0811 06:59:07.351876 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.351880 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.351941 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.352044 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.352894 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.353597 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.354766 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.355233 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.355377 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.358493 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.358500 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.358505 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.358544 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.361870 (+ 3326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.362533 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.362542 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.362547 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.362719 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.363093 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.363099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.363104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.363542 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.363715 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.364082 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.364383 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 06:59:07.365901 28506 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } has no permanent_uuid. Determining permanent_uuid...
I20260811 06:59:07.367026 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.366657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50136)
0811 06:59:07.366811 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.366895 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:59:07.368045 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.367684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50140)
0811 06:59:07.367841 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.367921 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 06:59:07.368266 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.419509 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 1
I20260811 06:59:07.420848 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.420305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50150)
0811 06:59:07.420528 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.420638 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:59:07.421901 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.421536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50160)
0811 06:59:07.421690 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.421772 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 06:59:07.422122 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.431972 28655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 3
I20260811 06:59:07.433197 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.432613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50162)
0811 06:59:07.432791 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.432879 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 06:59:07.434463 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.433988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50176)
0811 06:59:07.434229 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.434322 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260811 06:59:07.434733 28655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.490360 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 2
I20260811 06:59:07.491652 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.491066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50184)
0811 06:59:07.491313 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.491442 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 06:59:07.492826 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.492360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50190)
0811 06:59:07.492568 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.492665 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 06:59:07.493088 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.577971 28655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 4
I20260811 06:59:07.579156 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.578631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50198)
0811 06:59:07.578880 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.578984 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 06:59:07.580348 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.579999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50204)
0811 06:59:07.580183 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.580249 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 06:59:07.580606 28655 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
W20260811 06:59:07.589679 28667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:07.589974 28667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:07.590137 28667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:07.597430 28667 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:07.597551 28667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:07.597651 28667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:07.597746 28667 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:07.597815 28667 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:07.599324 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 3
I20260811 06:59:07.600508 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.599920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50218)
0811 06:59:07.600207 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.600309 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 06:59:07.601717 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.601237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50224)
0811 06:59:07.601426 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.601556 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260811 06:59:07.601994 28506 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:46839: Network error: Client connection negotiation failed: client connection to 127.4.40.60:46839: connect: Connection refused (error 111)
I20260811 06:59:07.608402 28667 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36905,127.4.40.61:41631,127.4.40.60:46839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab
--principal=kudu/127.4.40.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:46839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.40.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:07.609673 28667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:07.611016 28667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:07.623827 28679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:07.623777 28681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:07.623865 28676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:07.624655 28667 server_base.cc:1034] running on GCE node
I20260811 06:59:07.625581 28667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:07.627022 28667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:07.628203 28667 hybrid_clock.cc:648] HybridClock initialized: now 1786431547628138 us; error 43 us; skew 500 ppm
I20260811 06:59:07.628620 28667 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:07.631595 28667 init.cc:377] Logged in from keytab as kudu/127.4.40.60@KRBTEST.COM (short username kudu)
I20260811 06:59:07.633010 28667 webserver.cc:460] Webserver started at http://127.4.40.60:43347/ using document root <none> and password file <none>
I20260811 06:59:07.633703 28667 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:07.633853 28667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:07.634167 28667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:07.637245 28667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data/instance:
uuid: "1297a42617f646acbae4de2a88d1400e"
format_stamp: "Formatted at 2026-08-11 06:59:07 on dist-test-slave-7vjv"
server_key: "4643fae841a83555330116efb49ff032"
server_key_iv: "e9ea6a655c22d74988f9d3094294ff45"
server_key_version: "encryptionkey@0"
I20260811 06:59:07.638052 28667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal/instance:
uuid: "1297a42617f646acbae4de2a88d1400e"
format_stamp: "Formatted at 2026-08-11 06:59:07 on dist-test-slave-7vjv"
server_key: "4643fae841a83555330116efb49ff032"
server_key_iv: "e9ea6a655c22d74988f9d3094294ff45"
server_key_version: "encryptionkey@0"
I20260811 06:59:07.643404 28667 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 06:59:07.647099 28690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:07.648588 28667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:59:07.648788 28667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal
uuid: "1297a42617f646acbae4de2a88d1400e"
format_stamp: "Formatted at 2026-08-11 06:59:07 on dist-test-slave-7vjv"
server_key: "4643fae841a83555330116efb49ff032"
server_key_iv: "e9ea6a655c22d74988f9d3094294ff45"
server_key_version: "encryptionkey@0"
I20260811 06:59:07.648993 28667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:07.670292 28667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:07.674263 28667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:07.674556 28667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:07.693343 28667 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.60:46839
I20260811 06:59:07.693375 28785 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.60:46839 every 8 connection(s)
I20260811 06:59:07.694682 28667 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
I20260811 06:59:07.697784 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 28667
I20260811 06:59:07.698001 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786431333861527-4256-0/minicluster-data/master-2/wal/instance
I20260811 06:59:07.698521 4256 external_mini_cluster.cc:1428] Setting key 6c69d0c26b821f7f192b3cc59eb5da18
I20260811 06:59:07.698642 28786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:07.708722 28786 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:07.722502 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.700447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50230 (local address 127.4.40.60:46839)
0811 06:59:07.701296 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.701312 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.701370 (+ 58us) server_negotiation.cc:407] Connection header received
0811 06:59:07.703675 (+ 2305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.703702 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.704201 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.704644 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.705927 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.707877 (+ 1950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.709322 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.710320 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.710495 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.715154 (+ 4659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.715162 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.715171 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.715213 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.718460 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.719166 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.719174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.719179 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.719335 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.719840 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.719844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.719846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.720405 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.720731 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.721174 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.721581 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":384,"threads_started":1}
I20260811 06:59:07.726905 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.711409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:36905 (local address 127.0.0.1:34360)
0811 06:59:07.712297 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.712331 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.712505 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.712890 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.712897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.713602 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.713882 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.713890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.715016 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.715020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.716351 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.716363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.717314 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.717322 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.717481 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.718093 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.718109 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.720266 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.723522 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.723534 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.723540 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.724198 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.724589 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.724593 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.724596 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.724759 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.725979 (+ 1220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.725985 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.726115 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.726246 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.726492 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":496,"threads_started":1}
I20260811 06:59:07.727041 28790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.711367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34360 (local address 127.4.40.62:36905)
0811 06:59:07.711966 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.711973 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.712433 (+ 460us) server_negotiation.cc:407] Connection header received
0811 06:59:07.712629 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.712646 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.712711 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.712821 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.714039 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.714817 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.716508 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.717107 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.717258 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.720452 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.720457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.720461 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.720493 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.723386 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.724340 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.724344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.724347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.724472 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.725259 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.725263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.725282 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.725693 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.725882 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.726410 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.726665 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":312,"threads_started":1}
I20260811 06:59:07.728526 28786 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 06:59:07.741145 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.725569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34366 (local address 127.4.40.62:36905)
0811 06:59:07.726192 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.726196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.726215 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:59:07.726282 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.726288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.726351 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.726460 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.728128 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.729261 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.730582 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.731012 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.731147 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.735290 (+ 4143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.735295 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.735298 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.735326 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.738277 (+ 2951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.739114 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.739118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.739120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.739233 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.739649 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.739656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.739659 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.739929 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.740250 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.740527 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.740744 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":330,"threads_started":1}
I20260811 06:59:07.741762 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.729305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:41631 (local address 127.0.0.1:55098)
0811 06:59:07.729520 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.729538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.729674 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.729999 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.730005 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.730343 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.730479 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.730486 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.731408 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.731413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.732452 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.732465 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.733188 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.733193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.733374 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.733899 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.733915 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.735929 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.739346 (+ 3417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.739351 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.739355 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.739731 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.740139 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.740145 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.740149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.740296 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.740800 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.740805 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.740986 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.741144 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.741411 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129}
I20260811 06:59:07.741829 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.729319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55098 (local address 127.4.40.61:41631)
0811 06:59:07.729516 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.729522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.729618 (+ 96us) server_negotiation.cc:407] Connection header received
0811 06:59:07.729787 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.729792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.729864 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.729964 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.730631 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.731264 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.732600 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.733034 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.733177 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.736173 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.736187 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.736191 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.736224 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.739220 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.739859 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.739865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.739868 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.740007 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.740411 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.740419 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.740422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.740677 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.740832 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.741237 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.741447 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:59:07.742810 28786 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 06:59:07.756815 28791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.743579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50246)
0811 06:59:07.743729 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.743749 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.743855 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.744175 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.744180 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.744522 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.744646 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.744654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.745911 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.745916 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.747114 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.747126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.748064 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.748068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.748272 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.748826 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.748842 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.751121 (+ 2279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.754246 (+ 3125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.754253 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.754257 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.754610 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.755151 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.755157 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.755160 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.755288 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.755757 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.755762 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.755895 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.756073 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.756390 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 06:59:07.756985 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.743554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50246 (local address 127.4.40.60:46839)
0811 06:59:07.743714 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.743718 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.743820 (+ 102us) server_negotiation.cc:407] Connection header received
0811 06:59:07.743948 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.743951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.743999 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.744074 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.744797 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.745784 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.747256 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.747903 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.748028 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.751287 (+ 3259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.751291 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.751294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.751322 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.754116 (+ 2794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.754912 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.754915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.754917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.755035 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.755391 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.755397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.755400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.755646 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.755793 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.756178 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.756554 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":53}
I20260811 06:59:07.758894 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.743019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55108 (local address 127.4.40.61:41631)
0811 06:59:07.743178 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.743185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.743772 (+ 587us) server_negotiation.cc:407] Connection header received
0811 06:59:07.743828 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.743832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.743900 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.743997 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.745604 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.746584 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.748237 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.748658 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.748796 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.753375 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.753381 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.753385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.753424 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.755846 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.756595 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.756600 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.756603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.756751 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.757208 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.757215 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.757219 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.757577 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.757862 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.758165 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.758393 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:59:07.763307 28786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: Bootstrap starting.
I20260811 06:59:07.764262 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:59:07.767295 28786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:07.768752 28786 log.cc:824] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:07.772462 28786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: No bootstrap required, opened a new log
I20260811 06:59:07.779207 28506 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 4
I20260811 06:59:07.790403 28795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.770264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34374 (local address 127.4.40.62:36905)
0811 06:59:07.770732 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.770740 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.771287 (+ 547us) server_negotiation.cc:407] Connection header received
0811 06:59:07.771480 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.771487 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.771570 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.771670 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.774789 (+ 3119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.776198 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.778666 (+ 2468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.779264 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.779376 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.783325 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.783333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.783337 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.783378 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.786982 (+ 3604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.787910 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.787918 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.787922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.788081 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.788592 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.788597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.788600 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.788957 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.789228 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.789830 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.790049 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284}
I20260811 06:59:07.793243 28786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:07.793717 28786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:07.793902 28786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1297a42617f646acbae4de2a88d1400e, State: Initialized, Role: FOLLOWER
I20260811 06:59:07.794818 28786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431546, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 06:59:07.801471 28789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.770602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50258 (local address 127.4.40.60:46839)
0811 06:59:07.773765 (+ 3163us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.773771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.778345 (+ 4574us) server_negotiation.cc:407] Connection header received
0811 06:59:07.778793 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.778798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.778862 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.778921 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.781977 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.783079 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.785369 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.786167 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.786266 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.792906 (+ 6640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.792912 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.792915 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.792949 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.796829 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.797882 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.797890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.797895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.798084 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.798649 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.798657 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.798662 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.799059 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.799315 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.799886 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.800220 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3018}
I20260811 06:59:07.802242 28786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:07.802632 28811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:07.803777 28811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:07.804641 28660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.770160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55110 (local address 127.4.40.61:41631)
0811 06:59:07.770493 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.770520 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.781591 (+ 11071us) server_negotiation.cc:407] Connection header received
0811 06:59:07.782135 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.782144 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.782237 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.782315 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.784915 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.786314 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.791053 (+ 4739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.791739 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.791845 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.796070 (+ 4225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.796076 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.796080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.796143 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.800468 (+ 4325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.801347 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.801354 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.801360 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.801549 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.802124 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.802133 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.802136 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.802649 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.803856 (+ 1207us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.804012 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.804278 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260811 06:59:07.804699 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.779834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50268)
0811 06:59:07.780629 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.780654 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.780797 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.787852 (+ 7055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.787858 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.788366 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.788578 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.788589 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.790429 (+ 1840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.790434 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.791873 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.791888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.792948 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.792956 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.793147 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.793712 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.793736 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.796625 (+ 2889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.801786 (+ 5161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.801791 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.801794 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.802315 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.802792 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.802797 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.802800 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.802972 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.803695 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.803700 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.803876 (+ 176us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.804020 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.804359 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637}
I20260811 06:59:07.805871 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.780226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.4.40.60:46839)
0811 06:59:07.787488 (+ 7262us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.787497 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.787531 (+ 34us) server_negotiation.cc:407] Connection header received
0811 06:59:07.787584 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.787588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.787659 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.788726 (+ 1067us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.789003 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.790263 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.792066 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.792732 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.793460 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.796983 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.796991 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.796998 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.797036 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.801602 (+ 4566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.802463 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.802472 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.802475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.802643 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.803123 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.803128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.803131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.803539 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.805056 (+ 1517us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.805179 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.805433 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7132,"spinlock_wait_cycles":19200,"thread_start_us":10435,"threads_started":1}
I20260811 06:59:07.811527 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: Bootstrap starting.
W20260811 06:59:07.817709 28833 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:59:07.817925 28833 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:59:07.820168 28506 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:07.821573 28506 log.cc:824] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:07.825057 28506 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: No bootstrap required, opened a new log
I20260811 06:59:07.813627 28667 master_runner.cc:186] Error getting master registration for 127.4.40.62:36905: OK, instance_id { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" instance_seqno: 1786431546636892 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:59:07.825433 28667 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 06:59:07.832188 28506 raft_consensus.cc:348] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:07.832592 28506 raft_consensus.cc:374] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:07.832727 28506 raft_consensus.cc:729] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 654b4770e4034816ae6f7b2e9a5716db, State: Initialized, Role: FOLLOWER
I20260811 06:59:07.833369 28506 consensus_queue.cc:260] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:07.834318 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:07.834726 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:07.835496 28506 sys_catalog.cc:564] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:59:07.847141 28857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:59:07.847326 28857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 06:59:07.844987 28386 master_runner.cc:186] Error getting master registration for 127.4.40.61:41631: OK, instance_id { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" instance_seqno: 1786431547150122 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 06:59:07.854364 28386 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:59:07.876360 28424 tablet.cc:2369] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:07.879845 28655 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } attempt: 5
Aug 11 06:59:07 dist-test-slave-7vjv krb5kdc[28366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431547, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 06:59:07.894526 28659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.880451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:46839 (local address 127.0.0.1:50270)
0811 06:59:07.880708 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:07.880728 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:07.880890 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:07.881207 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:07.881212 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:07.881642 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:07.881814 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.881822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.882820 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.882826 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.883927 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:07.883939 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.884740 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.884746 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:07.884965 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.885571 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:07.885588 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:07.888001 (+ 2413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:07.891983 (+ 3982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.891988 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.891992 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.892428 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.892801 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:07.892806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:07.892809 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:07.892952 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:07.893510 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:07.893515 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:07.893668 (+ 153us) client_negotiation.cc:769] Sending connection context
0811 06:59:07.893814 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 06:59:07.894126 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136}
I20260811 06:59:07.894547 28808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:07.880486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.4.40.60:46839)
0811 06:59:07.880686 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:07.880692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:07.880822 (+ 130us) server_negotiation.cc:407] Connection header received
0811 06:59:07.881007 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:07.881012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:07.881078 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:07.881177 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:07.881955 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.882689 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.884074 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:07.884586 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:07.884741 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:07.888957 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:07.888962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:07.888967 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:07.889005 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:07.891833 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.892546 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.892553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.892556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.892690 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:07.893068 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:07.893074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:07.893077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:07.893383 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:07.893547 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:07.893931 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 06:59:07.894145 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 06:59:07.899395 28655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: Bootstrap starting.
I20260811 06:59:07.903450 28655 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:07.905220 28655 log.cc:824] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:07.908012 28655 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: No bootstrap required, opened a new log
I20260811 06:59:07.915702 28655 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:07.916123 28655 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:07.916289 28655 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24ab3ac21f224bb5970b3d66d896585e, State: Initialized, Role: FOLLOWER
I20260811 06:59:07.917083 28655 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:07.921080 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:07.921581 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:07.922331 28655 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 06:59:07.926692 28703 tablet.cc:2369] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:07.944487 28522 master_runner.cc:263] No leader master found from master 24ab3ac21f224bb5970b3d66d896585e
I20260811 06:59:07.944656 28522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 06:59:07.950874 28881 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 06:59:07.951104 28881 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 06:59:08.138968 28571 tablet.cc:2369] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:08.149852 28839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:59:08.150213 28839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:08.151889 28839 leader_election.cc:288] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24ab3ac21f224bb5970b3d66d896585e (127.4.40.61:41631), 1297a42617f646acbae4de2a88d1400e (127.4.40.60:46839)
I20260811 06:59:08.153487 28602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "654b4770e4034816ae6f7b2e9a5716db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24ab3ac21f224bb5970b3d66d896585e" is_pre_election: true
I20260811 06:59:08.154146 28602 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654b4770e4034816ae6f7b2e9a5716db in term 0.
I20260811 06:59:08.155089 28422 leader_election.cc:302] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24ab3ac21f224bb5970b3d66d896585e, 654b4770e4034816ae6f7b2e9a5716db; no voters:
I20260811 06:59:08.160234 28839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:59:08.160399 28839 raft_consensus.cc:482] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:59:08.160502 28839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:08.162415 28742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "654b4770e4034816ae6f7b2e9a5716db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1297a42617f646acbae4de2a88d1400e" is_pre_election: true
I20260811 06:59:08.163057 28742 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 654b4770e4034816ae6f7b2e9a5716db in term 0.
I20260811 06:59:08.176532 28839 raft_consensus.cc:504] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:08.178280 28602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "654b4770e4034816ae6f7b2e9a5716db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24ab3ac21f224bb5970b3d66d896585e"
I20260811 06:59:08.178532 28602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:08.184166 28602 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654b4770e4034816ae6f7b2e9a5716db in term 1.
I20260811 06:59:08.184737 28422 leader_election.cc:302] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24ab3ac21f224bb5970b3d66d896585e, 654b4770e4034816ae6f7b2e9a5716db; no voters:
I20260811 06:59:08.192929 28742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "654b4770e4034816ae6f7b2e9a5716db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1297a42617f646acbae4de2a88d1400e"
I20260811 06:59:08.193198 28742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:08.197098 28742 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 654b4770e4034816ae6f7b2e9a5716db in term 1.
I20260811 06:59:08.197921 28839 leader_election.cc:288] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [CANDIDATE]: Term 1 election: Requested vote from peers 24ab3ac21f224bb5970b3d66d896585e (127.4.40.61:41631), 1297a42617f646acbae4de2a88d1400e (127.4.40.60:46839)
I20260811 06:59:08.198374 28839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:08.198926 28839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [term 1 LEADER]: Becoming Leader. State: Replica: 654b4770e4034816ae6f7b2e9a5716db, State: Running, Role: LEADER
I20260811 06:59:08.201400 28839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } }
I20260811 06:59:08.223755 28885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: SysCatalogTable state changed. Reason: New leader 654b4770e4034816ae6f7b2e9a5716db. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.224531 28885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:08.226887 28890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:08.229326 28890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:08.249428 28742 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [term 1 FOLLOWER]: Refusing update from remote peer 654b4770e4034816ae6f7b2e9a5716db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:08.250702 28885 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [LEADER]: Connected to new peer: Peer: permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:08.251497 28602 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [term 1 FOLLOWER]: Refusing update from remote peer 654b4770e4034816ae6f7b2e9a5716db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:08.256258 28885 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [LEADER]: Connected to new peer: Peer: permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:08.289682 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 654b4770e4034816ae6f7b2e9a5716db. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.290078 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:08.297307 28891 mvcc.cc:205] Tried to move back new op lower bound from 7317223621618151424 to 7317223621446897664. Current Snapshot: MvccSnapshot[applied={T|T < 7317223621618151424}]
I20260811 06:59:08.298491 28811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 654b4770e4034816ae6f7b2e9a5716db. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.298851 28811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:08.301520 28893 mvcc.cc:205] Tried to move back new op lower bound from 7317223621618151424 to 7317223621446897664. Current Snapshot: MvccSnapshot[applied={T|T < 7317223621618151424}]
I20260811 06:59:08.309195 28892 mvcc.cc:205] Tried to move back new op lower bound from 7317223621618151424 to 7317223621446897664. Current Snapshot: MvccSnapshot[applied={T|T < 7317223621618151424}]
I20260811 06:59:08.312391 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.312528 28868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.312752 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:08.312844 28868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:08.312991 28839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.313263 28839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:08.314169 28890 catalog_manager.cc:1347] Generated new cluster ID: ed5a6b2eb85245299eeed5229b6e8368
I20260811 06:59:08.314272 28890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:59:08.315992 28811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "654b4770e4034816ae6f7b2e9a5716db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "654b4770e4034816ae6f7b2e9a5716db" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36905 } } peers { permanent_uuid: "24ab3ac21f224bb5970b3d66d896585e" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 41631 } } peers { permanent_uuid: "1297a42617f646acbae4de2a88d1400e" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 46839 } } }
I20260811 06:59:08.316303 28811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 06:59:08.348119 28890 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:59:08.350389 28890 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:59:08.369297 28890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 654b4770e4034816ae6f7b2e9a5716db: Generated new TSK 0
I20260811 06:59:08.370579 28890 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:59:08.820004 28833 catalog_manager.cc:1259] Loaded cluster ID: ed5a6b2eb85245299eeed5229b6e8368
I20260811 06:59:08.820154 28833 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: loading cluster ID for follower catalog manager: success
I20260811 06:59:08.822777 28833 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: acquiring CA information for follower catalog manager: success
I20260811 06:59:08.824703 28833 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1297a42617f646acbae4de2a88d1400e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:59:08.953426 28881 catalog_manager.cc:1259] Loaded cluster ID: ed5a6b2eb85245299eeed5229b6e8368
I20260811 06:59:08.953536 28881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: loading cluster ID for follower catalog manager: success
I20260811 06:59:08.955926 28881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: acquiring CA information for follower catalog manager: success
I20260811 06:59:08.957670 28881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24ab3ac21f224bb5970b3d66d896585e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 06:59:08.966125 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 28386
I20260811 06:59:08.967465 28493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:09.078537 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 28386
I20260811 06:59:09.108153 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 28522
I20260811 06:59:09.109342 28642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:09.216998 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 28522
I20260811 06:59:09.249642 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 28667
I20260811 06:59:09.250808 28780 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:09.359503 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 28667
2026-08-11T06:59:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4883 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:59:09 dist-test-slave-7vjv krb5kdc[28921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:59:09 dist-test-slave-7vjv krb5kdc[28921](info): set up 2 sockets
Aug 11 06:59:09 dist-test-slave-7vjv krb5kdc[28921](info): commencing operation
krb5kdc: starting...
W20260811 06:59:10.746341 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.336s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:59:10 dist-test-slave-7vjv krb5kdc[28921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:59:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:59:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:59:11.035651 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37715
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:33527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:37715
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:11.370472 28942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:11.370793 28942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:11.370955 28942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:11.377874 28942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:11.378013 28942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:11.378120 28942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:11.378219 28942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:11.378309 28942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:11.387740 28942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:37715
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37715
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:11.389122 28942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:11.390506 28942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:11.402185 28952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:11.402350 28953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:11.402153 28957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:11.404065 28942 server_base.cc:1034] running on GCE node
I20260811 06:59:11.404912 28942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:11.406198 28942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:11.407407 28942 hybrid_clock.cc:648] HybridClock initialized: now 1786431551407367 us; error 31 us; skew 500 ppm
I20260811 06:59:11.407752 28942 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:11 dist-test-slave-7vjv krb5kdc[28921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431551, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:11.410864 28942 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:11.412343 28942 webserver.cc:460] Webserver started at http://127.4.40.62:39555/ using document root <none> and password file <none>
I20260811 06:59:11.412967 28942 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:11.413076 28942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:11.413309 28942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:11.416168 28942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "6fc4306159bf4aa2b1d9e8523dce9111"
format_stamp: "Formatted at 2026-08-11 06:59:11 on dist-test-slave-7vjv"
server_key: "be36c10b89bd30a951f401768d935172"
server_key_iv: "0918f1b4f10d2503bfa9038fd7007d7c"
server_key_version: "encryptionkey@0"
I20260811 06:59:11.416894 28942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "6fc4306159bf4aa2b1d9e8523dce9111"
format_stamp: "Formatted at 2026-08-11 06:59:11 on dist-test-slave-7vjv"
server_key: "be36c10b89bd30a951f401768d935172"
server_key_iv: "0918f1b4f10d2503bfa9038fd7007d7c"
server_key_version: "encryptionkey@0"
I20260811 06:59:11.421983 28942 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 06:59:11.425318 28968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:11.426594 28942 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:59:11.426781 28942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "6fc4306159bf4aa2b1d9e8523dce9111"
format_stamp: "Formatted at 2026-08-11 06:59:11 on dist-test-slave-7vjv"
server_key: "be36c10b89bd30a951f401768d935172"
server_key_iv: "0918f1b4f10d2503bfa9038fd7007d7c"
server_key_version: "encryptionkey@0"
I20260811 06:59:11.426970 28942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:11.456664 28942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:11.461092 28942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:11.461419 28942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:11.480715 28942 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:37715
I20260811 06:59:11.480749 29080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:37715 every 8 connection(s)
I20260811 06:59:11.482111 28942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:11.483232 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 28942
I20260811 06:59:11.483500 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:59:11.484282 4256 external_mini_cluster.cc:1428] Setting key 941ceb21a3971a837bde2b5ca7b97b58
I20260811 06:59:11.486574 29081 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 06:59:11 dist-test-slave-7vjv krb5kdc[28921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:11.499935 29081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111: Bootstrap starting.
I20260811 06:59:11.504176 29081 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:11.505434 29081 log.cc:824] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:11.508831 29081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111: No bootstrap required, opened a new log
I20260811 06:59:11.509136 29087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:11.486177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39540 (local address 127.4.40.62:37715)
0811 06:59:11.487007 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:11.487018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:11.487059 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:59:11.488789 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:11.488809 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:11.489207 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:11.489543 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:11.491534 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:11.492933 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:11.494814 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:11.495429 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:11.495560 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:11.500606 (+ 5046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:11.500617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:11.500633 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:11.500675 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:11.504682 (+ 4007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:11.505406 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:11.505416 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:11.505430 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:11.505655 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:11.506132 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:11.506136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:11.506139 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:11.506675 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:11.507022 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:11.507411 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 06:59:11.507834 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":381,"threads_started":1}
I20260811 06:59:11.513016 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 06:59:11.517319 29081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37715 } }
I20260811 06:59:11.517683 29081 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:11.517822 29081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc4306159bf4aa2b1d9e8523dce9111, State: Initialized, Role: FOLLOWER
I20260811 06:59:11.518343 29081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37715 } }
I20260811 06:59:11.518595 29081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:59:11.518759 29081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:59:11.518934 29081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:11.521171 29081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37715 } }
I20260811 06:59:11.521692 29081 leader_election.cc:302] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fc4306159bf4aa2b1d9e8523dce9111; no voters:
I20260811 06:59:11.522349 29081 leader_election.cc:288] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:59:11.522543 29089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:11.523383 29089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [term 1 LEADER]: Becoming Leader. State: Replica: 6fc4306159bf4aa2b1d9e8523dce9111, State: Running, Role: LEADER
I20260811 06:59:11.523944 29089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37715 } }
I20260811 06:59:11.524688 29081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:11.526832 29091 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fc4306159bf4aa2b1d9e8523dce9111. Latest consensus state: current_term: 1 leader_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37715 } } }
I20260811 06:59:11.527307 29091 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:11.527894 29090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fc4306159bf4aa2b1d9e8523dce9111" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37715 } } }
I20260811 06:59:11.528245 29090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:11.533227 29109 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:11.538719 29109 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:11.547345 29109 catalog_manager.cc:1347] Generated new cluster ID: 53e31d67ee734a0c8f09ccdf6fe7b64e
I20260811 06:59:11.547477 29109 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:59:11.574584 29109 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:59:11.575898 29109 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:59:11.586956 29109 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fc4306159bf4aa2b1d9e8523dce9111: Generated new TSK 0
I20260811 06:59:11.587746 29109 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 06:59:11.624969 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 28942
I20260811 06:59:11.626106 29074 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:11.721391 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 28942
2026-08-11T06:59:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2358 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:59:11 dist-test-slave-7vjv krb5kdc[29134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:59:11 dist-test-slave-7vjv krb5kdc[29134](info): set up 2 sockets
Aug 11 06:59:11 dist-test-slave-7vjv krb5kdc[29134](info): commencing operation
krb5kdc: starting...
W20260811 06:59:13.089517 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:59:13 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:59:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:59:13.384711 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37531
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:37531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:13.747622 29160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:13.747921 29160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:13.748009 29160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:13.755041 29160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:13.755139 29160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:13.755183 29160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:13.755239 29160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:13.755282 29160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:13.765352 29160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:37531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:13.766572 29160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:13.767940 29160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:13.779495 29173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:13.779541 29171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:13.779584 29170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:13.780239 29160 server_base.cc:1034] running on GCE node
I20260811 06:59:13.780970 29160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:13.782717 29160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:13.783939 29160 hybrid_clock.cc:648] HybridClock initialized: now 1786431553783872 us; error 52 us; skew 500 ppm
I20260811 06:59:13.784365 29160 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:13 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431553, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:13.787626 29160 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:13.789104 29160 webserver.cc:460] Webserver started at http://127.4.40.62:35803/ using document root <none> and password file <none>
I20260811 06:59:13.789736 29160 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:13.789837 29160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:13.790109 29160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:13.793123 29160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "561ef81780f04a80822de7800689a027"
format_stamp: "Formatted at 2026-08-11 06:59:13 on dist-test-slave-7vjv"
server_key: "964f28137fbabba0f7906170a1deaae0"
server_key_iv: "cd263cb78baa4bf020c00bdb31a35da6"
server_key_version: "encryptionkey@0"
I20260811 06:59:13.793901 29160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "561ef81780f04a80822de7800689a027"
format_stamp: "Formatted at 2026-08-11 06:59:13 on dist-test-slave-7vjv"
server_key: "964f28137fbabba0f7906170a1deaae0"
server_key_iv: "cd263cb78baa4bf020c00bdb31a35da6"
server_key_version: "encryptionkey@0"
I20260811 06:59:13.799057 29160 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 06:59:13.802414 29185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:13.803715 29160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 06:59:13.803850 29160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "561ef81780f04a80822de7800689a027"
format_stamp: "Formatted at 2026-08-11 06:59:13 on dist-test-slave-7vjv"
server_key: "964f28137fbabba0f7906170a1deaae0"
server_key_iv: "cd263cb78baa4bf020c00bdb31a35da6"
server_key_version: "encryptionkey@0"
I20260811 06:59:13.804077 29160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:13.826419 29160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:13.830552 29160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:13.830780 29160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:13.849812 29160 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:37531
I20260811 06:59:13.849853 29283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:37531 every 8 connection(s)
I20260811 06:59:13.851142 29160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:13.854729 29285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:13.855208 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 29160
I20260811 06:59:13.855432 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:59:13.856215 4256 external_mini_cluster.cc:1428] Setting key bc6502395590918addba4b5a8bf480ca
I20260811 06:59:13.868709 29285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap starting.
Aug 11 06:59:13 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:13.871724 29285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:13.872943 29285 log.cc:824] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:13.876446 29285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: No bootstrap required, opened a new log
I20260811 06:59:13.880683 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:13.858231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.4.40.62:37531)
0811 06:59:13.859095 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:13.859109 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:13.859159 (+ 50us) server_negotiation.cc:407] Connection header received
0811 06:59:13.861342 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:13.861368 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:13.861837 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:13.862282 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:13.863652 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:13.865790 (+ 2138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:13.867115 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:13.868145 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:13.868325 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:13.872368 (+ 4043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:13.872378 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:13.872394 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:13.872435 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:13.876329 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:13.877014 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:13.877023 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:13.877034 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:13.877251 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:13.877607 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:13.877615 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:13.877622 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:13.878280 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:13.878599 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:13.878886 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 06:59:13.879318 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":348,"threads_started":1}
I20260811 06:59:13.885453 29285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:13.885798 29285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:13.885870 29285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561ef81780f04a80822de7800689a027, State: Initialized, Role: FOLLOWER
I20260811 06:59:13.886332 29285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:13.886528 29285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:59:13.886627 29285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:59:13.886742 29285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:13.889022 29285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:13.889570 29285 leader_election.cc:302] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 561ef81780f04a80822de7800689a027; no voters:
I20260811 06:59:13.890203 29285 leader_election.cc:288] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:59:13.890336 29295 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:13.891023 29295 raft_consensus.cc:686] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 LEADER]: Becoming Leader. State: Replica: 561ef81780f04a80822de7800689a027, State: Running, Role: LEADER
I20260811 06:59:13.891628 29295 consensus_queue.cc:237] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:13.892277 29285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:13.895131 29299 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 561ef81780f04a80822de7800689a027. Latest consensus state: current_term: 1 leader_uuid: "561ef81780f04a80822de7800689a027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } } }
I20260811 06:59:13.895500 29299 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:13.895407 29298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "561ef81780f04a80822de7800689a027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } } }
I20260811 06:59:13.895741 29298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:13.897732 29312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:13.902915 29312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:13.912142 29312 catalog_manager.cc:1347] Generated new cluster ID: c4550c75cf934703a5508e9d5130d86f
I20260811 06:59:13.912254 29312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:59:13.941682 29312 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:59:13.943614 29312 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:59:13.967039 29312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Generated new TSK 0
I20260811 06:59:13.968247 29312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:59:14.044489 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37531
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:14.391194 29337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:14.391530 29337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:14.391700 29337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:14.399423 29337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:14.399564 29337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:14.399781 29337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:59:14.409960 29337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:14.411298 29337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:14.412721 29337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:14.426102 29343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:14.426154 29348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:14.426186 29344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:14.427017 29337 server_base.cc:1034] running on GCE node
I20260811 06:59:14.427779 29337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:14.428748 29337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:14.429947 29337 hybrid_clock.cc:648] HybridClock initialized: now 1786431554429887 us; error 49 us; skew 500 ppm
I20260811 06:59:14.430251 29337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:14 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431554, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:14.433357 29337 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:59:14.434738 29337 webserver.cc:460] Webserver started at http://127.4.40.1:39575/ using document root <none> and password file <none>
I20260811 06:59:14.435432 29337 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:14.435575 29337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:14.435874 29337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:14.438743 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "118069c2bcab45d3b43d4ccd90454907"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "0b3e2cfe39d9d9314305bab1272b3fb5"
server_key_iv: "9d89f57c5e57c11598c54b34dc48e4fe"
server_key_version: "encryptionkey@0"
I20260811 06:59:14.439497 29337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "118069c2bcab45d3b43d4ccd90454907"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "0b3e2cfe39d9d9314305bab1272b3fb5"
server_key_iv: "9d89f57c5e57c11598c54b34dc48e4fe"
server_key_version: "encryptionkey@0"
I20260811 06:59:14.444685 29337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 06:59:14.448237 29359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:14.449571 29337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 06:59:14.449781 29337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "118069c2bcab45d3b43d4ccd90454907"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "0b3e2cfe39d9d9314305bab1272b3fb5"
server_key_iv: "9d89f57c5e57c11598c54b34dc48e4fe"
server_key_version: "encryptionkey@0"
I20260811 06:59:14.450075 29337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:14.470837 29337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:14.474982 29337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:14.475270 29337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:14.476347 29337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:14.477677 29337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:14.477780 29337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:14.477926 29337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:14.477996 29337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:14.512521 29337 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:33191
I20260811 06:59:14.512570 29562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:33191 every 8 connection(s)
I20260811 06:59:14.513813 29337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:59:14.524333 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 29337
I20260811 06:59:14.524557 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:59:14.525311 4256 external_mini_cluster.cc:1428] Setting key 211406d413f3f31b692f909b0d01159f
Aug 11 06:59:14 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431554, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:14.533351 29567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:14.516385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:45605 (local address 127.4.40.62:37531)
0811 06:59:14.516800 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:14.516804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:14.517726 (+ 922us) server_negotiation.cc:407] Connection header received
0811 06:59:14.519520 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:14.519526 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:14.519616 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:14.519753 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:14.521309 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:14.521971 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:14.523424 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:14.523920 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:14.524068 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:14.527777 (+ 3709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:14.527783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:14.527786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:14.527818 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:14.530436 (+ 2618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:14.531149 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:14.531159 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:14.531162 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:14.531277 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:14.531700 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:14.531704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:14.531707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:14.531954 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:14.532130 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:14.532785 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 06:59:14.532992 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":195,"threads_started":1}
I20260811 06:59:14.534291 29568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:14.516909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:45605)
0811 06:59:14.517575 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:14.517611 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:14.519246 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:14.519988 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:14.519996 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:14.520566 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:14.521129 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:14.521147 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:14.522120 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:14.522123 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:14.523264 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:14.523276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:14.524189 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:14.524195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:14.524417 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:14.525191 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:14.525218 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:14.527555 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:14.530587 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:14.530595 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:14.530609 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:14.531022 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:14.531387 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:14.531392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:14.531395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:14.531579 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:14.532081 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:14.532089 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:14.532424 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 06:59:14.532671 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 06:59:14.533169 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":249,"threads_started":1}
I20260811 06:59:14.535961 29563 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37531
I20260811 06:59:14.536280 29563 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:14.536993 29563 heartbeater.cc:511] Master 127.4.40.62:37531 requested a full tablet report, sending...
I20260811 06:59:14.539194 29205 ts_manager.cc:194] Registered new tserver with Master: 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1:33191)
I20260811 06:59:14.540787 29205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:45605
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:59:14.588877 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37531
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:14.929536 29573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:14.929790 29573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:14.929975 29573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:14.937124 29573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:14.937227 29573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:14.937420 29573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:59:14.946940 29573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:14.948239 29573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:14.949551 29573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:14.962457 29594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:14.962613 29591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:14.962627 29588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:14.963586 29573 server_base.cc:1034] running on GCE node
I20260811 06:59:14.964267 29573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:14.965214 29573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:14.966455 29573 hybrid_clock.cc:648] HybridClock initialized: now 1786431554966387 us; error 58 us; skew 500 ppm
I20260811 06:59:14.966754 29573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:14 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431554, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:14.969815 29573 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:59:14.971215 29573 webserver.cc:460] Webserver started at http://127.4.40.2:46445/ using document root <none> and password file <none>
I20260811 06:59:14.971944 29573 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:14.972163 29573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:14.972463 29573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:14.975363 29573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "88c800786a9d47adaba18767a63ec4a9"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "96eb947b354ef77abf6a2ab595279187"
server_key_iv: "695f1e47a60cca2404f1841f82b8f2a2"
server_key_version: "encryptionkey@0"
I20260811 06:59:14.976207 29573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "88c800786a9d47adaba18767a63ec4a9"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "96eb947b354ef77abf6a2ab595279187"
server_key_iv: "695f1e47a60cca2404f1841f82b8f2a2"
server_key_version: "encryptionkey@0"
I20260811 06:59:14.981221 29573 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 06:59:14.984493 29610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:14.985740 29573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 06:59:14.985903 29573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "88c800786a9d47adaba18767a63ec4a9"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "96eb947b354ef77abf6a2ab595279187"
server_key_iv: "695f1e47a60cca2404f1841f82b8f2a2"
server_key_version: "encryptionkey@0"
I20260811 06:59:14.986104 29573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:15.014961 29573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:15.018891 29573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:15.019203 29573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:15.020265 29573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:15.021713 29573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:15.021814 29573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:15.021955 29573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:15.022044 29573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:15.055840 29573 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:38107
I20260811 06:59:15.055891 29808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:38107 every 8 connection(s)
I20260811 06:59:15.057137 29573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:59:15.066892 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 29573
I20260811 06:59:15.067099 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:59:15.067672 4256 external_mini_cluster.cc:1428] Setting key bcc1be511f64dd509540009fbf0dbbad
Aug 11 06:59:15 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431554, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:15.077713 29811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.059614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:49543 (local address 127.4.40.62:37531)
0811 06:59:15.060064 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.060068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.061015 (+ 947us) server_negotiation.cc:407] Connection header received
0811 06:59:15.062800 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.062806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.062891 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.063016 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:15.064492 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.065221 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.066854 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.067430 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.067613 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.071834 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:15.071839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:15.071842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:15.071874 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:15.074645 (+ 2771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:15.075371 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:15.075376 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:15.075378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:15.075505 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:15.075947 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:15.075950 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:15.075952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:15.076286 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:15.076438 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.077117 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.077338 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":223,"threads_started":1}
I20260811 06:59:15.078658 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.060184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:49543)
0811 06:59:15.060849 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.060892 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.062533 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.063255 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.063262 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.063726 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:15.064304 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.064320 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.065383 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.065387 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.066623 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.066642 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.067944 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.067954 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.068362 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.069116 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:15.069138 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:15.071618 (+ 2480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:15.074804 (+ 3186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:15.074812 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:15.074827 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:15.075240 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:15.075620 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:15.075625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:15.075629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:15.075821 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:15.076425 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:15.076432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:15.076757 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.077008 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.077534 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":272,"threads_started":1}
I20260811 06:59:15.080087 29809 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37531
I20260811 06:59:15.080441 29809 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:15.081307 29809 heartbeater.cc:511] Master 127.4.40.62:37531 requested a full tablet report, sending...
I20260811 06:59:15.082916 29205 ts_manager.cc:194] Registered new tserver with Master: 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107)
I20260811 06:59:15.084192 29205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:49543
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:59:15.131850 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37531
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:15.466784 29818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:15.467077 29818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:15.467202 29818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:15.474275 29818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:15.474372 29818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:15.474550 29818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:59:15.484141 29818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:15.485517 29818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:15.486896 29818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:15.500734 29836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:15.500782 29830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:15.501106 29829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:15.501981 29818 server_base.cc:1034] running on GCE node
I20260811 06:59:15.502635 29818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:15.503536 29818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:15.504756 29818 hybrid_clock.cc:648] HybridClock initialized: now 1786431555504698 us; error 48 us; skew 500 ppm
I20260811 06:59:15.505084 29818 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:15 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:15.508348 29818 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:59:15.509722 29818 webserver.cc:460] Webserver started at http://127.4.40.3:38269/ using document root <none> and password file <none>
I20260811 06:59:15.510371 29818 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:15.510471 29818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:15.510720 29818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:15.513680 29818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "23763a99124c40fa982e4ffa1c5fe29f"
format_stamp: "Formatted at 2026-08-11 06:59:15 on dist-test-slave-7vjv"
server_key: "3ec70d95a1a04908ede8305dea97b2ab"
server_key_iv: "650213d819f8e6e275c2b871480772cd"
server_key_version: "encryptionkey@0"
I20260811 06:59:15.514451 29818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "23763a99124c40fa982e4ffa1c5fe29f"
format_stamp: "Formatted at 2026-08-11 06:59:15 on dist-test-slave-7vjv"
server_key: "3ec70d95a1a04908ede8305dea97b2ab"
server_key_iv: "650213d819f8e6e275c2b871480772cd"
server_key_version: "encryptionkey@0"
I20260811 06:59:15.519591 29818 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 06:59:15.522972 29848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:15.524305 29818 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:59:15.524458 29818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "23763a99124c40fa982e4ffa1c5fe29f"
format_stamp: "Formatted at 2026-08-11 06:59:15 on dist-test-slave-7vjv"
server_key: "3ec70d95a1a04908ede8305dea97b2ab"
server_key_iv: "650213d819f8e6e275c2b871480772cd"
server_key_version: "encryptionkey@0"
I20260811 06:59:15.524660 29818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:15.543550 29563 heartbeater.cc:503] Master 127.4.40.62:37531 was elected leader, sending a full tablet report...
I20260811 06:59:15.547408 29818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:15.551717 29818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:15.552014 29818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:15.553013 29818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:15.554394 29818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:15.554493 29818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:15.554615 29818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:15.554666 29818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:15.587860 29818 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:36665
I20260811 06:59:15.587946 30051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:36665 every 8 connection(s)
I20260811 06:59:15.589366 29818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:59:15.599176 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 29818
I20260811 06:59:15.599400 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:59:15.600025 4256 external_mini_cluster.cc:1428] Setting key 14ed27bf8b8a6322c7c21a77c0bd9881
Aug 11 06:59:15 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431555, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:15.608916 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.591997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:36217 (local address 127.4.40.62:37531)
0811 06:59:15.592554 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.592560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.593518 (+ 958us) server_negotiation.cc:407] Connection header received
0811 06:59:15.595291 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.595296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.595361 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.595458 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:15.596931 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.597705 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.599316 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.599989 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.600155 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.603637 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:15.603641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:15.603644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:15.603704 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:15.606157 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:15.606844 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:15.606849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:15.606851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:15.606988 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:15.607376 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:15.607379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:15.607381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:15.607630 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:15.607735 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.608507 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.608659 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":263,"threads_started":1}
I20260811 06:59:15.609795 30057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.592657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:36217)
0811 06:59:15.593367 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.593411 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.595048 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.595656 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.595663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.596223 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:15.596778 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.596798 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.597835 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.597838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.599170 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.599182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.600239 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.600245 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.600433 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.601152 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:15.601174 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:15.603457 (+ 2283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:15.606300 (+ 2843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:15.606307 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:15.606317 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:15.606724 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:15.607082 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:15.607085 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:15.607087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:15.607281 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:15.607740 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:15.607746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:15.608156 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.608381 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.608789 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":292,"threads_started":1}
I20260811 06:59:15.611091 30054 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37531
I20260811 06:59:15.611369 30054 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:15.612021 30054 heartbeater.cc:511] Master 127.4.40.62:37531 requested a full tablet report, sending...
I20260811 06:59:15.613363 29205 ts_manager.cc:194] Registered new tserver with Master: 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3:36665)
I20260811 06:59:15.614398 29205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:36217
I20260811 06:59:15.615300 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:59:15.632228 30056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.620196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48354 (local address 127.4.40.62:37531)
0811 06:59:15.620348 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.620351 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.620754 (+ 403us) server_negotiation.cc:407] Connection header received
0811 06:59:15.620903 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.620907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.620968 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.621065 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:15.622443 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.623187 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.624524 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.624929 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.625030 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.627324 (+ 2294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:15.627328 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:15.627331 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:15.627359 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:15.629802 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:15.630541 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:15.630545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:15.630547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:15.630656 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:15.630956 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:15.630968 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:15.630970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:15.631290 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:15.631468 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.631711 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.631855 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 06:59:15.636143 29204 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:59:15.642886 29204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:59:15.672307 30081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.654762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:36665 (local address 127.0.0.1:34428)
0811 06:59:15.656581 (+ 1819us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.656601 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.656718 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.660873 (+ 4155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.660882 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.660915 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:15.661359 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.661374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.664167 (+ 2793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.664175 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.667321 (+ 3146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.667341 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.671236 (+ 3895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.671243 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.671485 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.671548 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.671688 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.671890 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1699,"thread_start_us":2386,"threads_started":1}
I20260811 06:59:15.673026 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.655141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.4.40.3:36665)
0811 06:59:15.659825 (+ 4684us) spinlock_profiling.cc:243] Waited 1.76 ms on lock 0x60c000066380. stack: 00007fb86289a66e 00007fb86289a430 00007fb8619215ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fb87577d775 00007fb86d794270 00007fb8628be358 00000000009da5bb 00007fb8628ddfd7 00007fb8628d9bea 00007fb8628decee 00007fb86934de4a 00007fb86934d377 00007fb86935e32d
0811 06:59:15.660163 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.660173 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.660211 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:59:15.660316 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.660328 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.660603 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.661598 (+ 995us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:15.661883 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.663919 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.667621 (+ 3702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.670652 (+ 3031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.670824 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.671033 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.671777 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.671948 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1596,"spinlock_wait_cycles":4051328,"thread_start_us":1810,"threads_started":1}
I20260811 06:59:15.677153 29930 tablet_service.cc:1467] Processing CreateTablet for tablet f321453a5765401f9441982978d1d451 (DEFAULT_TABLE table=test-table [id=c9ff7ba784b1405980611d3ee86b65af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:15.678431 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.656940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:38107 (local address 127.0.0.1:56478)
0811 06:59:15.658305 (+ 1365us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.658324 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.658448 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.670519 (+ 12071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.670526 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.670560 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:15.670985 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.671001 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.674328 (+ 3327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.674333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.676016 (+ 1683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.676029 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.677635 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.677642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.677820 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.677852 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.677956 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.678087 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1223,"thread_start_us":1074,"threads_started":1}
I20260811 06:59:15.678745 29930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f321453a5765401f9441982978d1d451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:15.678483 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.657107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.4.40.2:38107)
0811 06:59:15.669809 (+ 12702us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.669822 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.669863 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:59:15.669989 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.670001 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.670287 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.672642 (+ 2355us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:15.672880 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.674152 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.676206 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.677406 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.677571 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.677759 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.678016 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.678196 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12474,"thread_start_us":12712,"threads_started":1}
I20260811 06:59:15.679591 30076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.654509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59608)
0811 06:59:15.655371 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.655418 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.655576 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.672208 (+ 16632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.672216 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.672241 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:15.672610 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.672622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.674627 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.674632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.676831 (+ 2199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.676846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.678835 (+ 1989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.678842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.679012 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.679043 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.679138 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.679258 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":496,"threads_started":1}
I20260811 06:59:15.679719 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.654810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59608 (local address 127.4.40.1:33191)
0811 06:59:15.671609 (+ 16799us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.671619 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.671649 (+ 30us) server_negotiation.cc:407] Connection header received
0811 06:59:15.671751 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.671758 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.671996 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.672229 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:15.672838 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.674438 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.677019 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.678433 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.678566 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.678784 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.679219 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.679381 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16619,"thread_start_us":16437,"threads_started":1}
I20260811 06:59:15.683295 29445 tablet_service.cc:1467] Processing CreateTablet for tablet f321453a5765401f9441982978d1d451 (DEFAULT_TABLE table=test-table [id=c9ff7ba784b1405980611d3ee86b65af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:15.683203 29693 tablet_service.cc:1467] Processing CreateTablet for tablet f321453a5765401f9441982978d1d451 (DEFAULT_TABLE table=test-table [id=c9ff7ba784b1405980611d3ee86b65af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:15.684693 29445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f321453a5765401f9441982978d1d451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:15.684692 29693 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f321453a5765401f9441982978d1d451. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:15.690253 30089 tablet_bootstrap.cc:492] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Bootstrap starting.
I20260811 06:59:15.693125 30089 tablet_bootstrap.cc:654] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:15.694160 30089 log.cc:824] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:15.699134 30095 tablet_bootstrap.cc:492] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Bootstrap starting.
I20260811 06:59:15.701195 30096 tablet_bootstrap.cc:492] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Bootstrap starting.
I20260811 06:59:15.702646 30089 tablet_bootstrap.cc:492] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: No bootstrap required, opened a new log
I20260811 06:59:15.702904 30089 ts_tablet_manager.cc:1397] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260811 06:59:15.703317 30095 tablet_bootstrap.cc:654] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:15.704632 30096 tablet_bootstrap.cc:654] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:15.704684 30095 log.cc:824] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:15.705794 30096 log.cc:824] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:15.708011 30095 tablet_bootstrap.cc:492] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: No bootstrap required, opened a new log
I20260811 06:59:15.708317 30095 ts_tablet_manager.cc:1397] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 06:59:15.708760 30096 tablet_bootstrap.cc:492] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: No bootstrap required, opened a new log
I20260811 06:59:15.709004 30096 ts_tablet_manager.cc:1397] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 06:59:15.712693 30089 raft_consensus.cc:348] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.713131 30089 raft_consensus.cc:374] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:15.713229 30089 raft_consensus.cc:729] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23763a99124c40fa982e4ffa1c5fe29f, State: Initialized, Role: FOLLOWER
I20260811 06:59:15.713804 30089 consensus_queue.cc:260] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.714911 30054 heartbeater.cc:503] Master 127.4.40.62:37531 was elected leader, sending a full tablet report...
I20260811 06:59:15.715485 30089 ts_tablet_manager.cc:1428] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 06:59:15.715798 30095 raft_consensus.cc:348] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.716215 30095 raft_consensus.cc:374] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:15.716357 30095 raft_consensus.cc:729] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118069c2bcab45d3b43d4ccd90454907, State: Initialized, Role: FOLLOWER
I20260811 06:59:15.717016 30095 consensus_queue.cc:260] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.718655 30095 ts_tablet_manager.cc:1428] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 06:59:15.718684 30096 raft_consensus.cc:348] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.719049 30096 raft_consensus.cc:374] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:15.719159 30096 raft_consensus.cc:729] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c800786a9d47adaba18767a63ec4a9, State: Initialized, Role: FOLLOWER
I20260811 06:59:15.719853 30096 consensus_queue.cc:260] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.721441 29809 heartbeater.cc:503] Master 127.4.40.62:37531 was elected leader, sending a full tablet report...
I20260811 06:59:15.721992 30096 ts_tablet_manager.cc:1428] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260811 06:59:15.766225 29566 tablet.cc:2369] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:59:15.809201 29810 tablet.cc:2369] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:59:15.841009 30055 tablet.cc:2369] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:15.897646 30103 raft_consensus.cc:482] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:59:15.897943 30103 raft_consensus.cc:504] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.899803 30103 leader_election.cc:288] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3:36665), 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107)
I20260811 06:59:15.906040 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.900072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:54585 (local address 127.4.40.3:36665)
0811 06:59:15.900337 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.900344 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.900884 (+ 540us) server_negotiation.cc:407] Connection header received
0811 06:59:15.901135 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.901140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.901198 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.901292 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:15.901841 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.902575 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.904348 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.905103 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.905251 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.905352 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.905629 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.905747 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 06:59:15.906140 30107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.900058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:36665 (local address 127.4.40.1:54585)
0811 06:59:15.900772 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.900788 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.900971 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.901314 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.901323 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.901351 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:15.901697 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.901707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.902706 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.902713 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.904204 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.904216 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.905309 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.905315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.905467 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.905492 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.905591 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.905716 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":483,"threads_started":1}
I20260811 06:59:15.906708 30108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.900065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:38107 (local address 127.4.40.1:44047)
0811 06:59:15.901333 (+ 1268us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:15.901347 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:15.901523 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:15.901864 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:15.901868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:15.901891 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:15.902231 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.902240 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.903241 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.903246 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.904672 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:15.904685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.905877 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.905884 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:15.906031 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.906056 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:59:15.906171 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 06:59:15.906310 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":137,"thread_start_us":799,"threads_started":1}
I20260811 06:59:15.906919 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.900298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:44047 (local address 127.4.40.2:38107)
0811 06:59:15.900566 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.900575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.901438 (+ 863us) server_negotiation.cc:407] Connection header received
0811 06:59:15.901684 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.901690 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.901750 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.901846 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:15.902377 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.903090 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.904833 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.905643 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.905802 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.905903 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.906517 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.906665 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 06:59:15.907100 29970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f321453a5765401f9441982978d1d451" candidate_uuid: "118069c2bcab45d3b43d4ccd90454907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23763a99124c40fa982e4ffa1c5fe29f" is_pre_election: true
I20260811 06:59:15.907678 29970 raft_consensus.cc:2454] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 118069c2bcab45d3b43d4ccd90454907 in term 0.
I20260811 06:59:15.907931 29723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f321453a5765401f9441982978d1d451" candidate_uuid: "118069c2bcab45d3b43d4ccd90454907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c800786a9d47adaba18767a63ec4a9" is_pre_election: true
I20260811 06:59:15.908382 29363 leader_election.cc:302] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118069c2bcab45d3b43d4ccd90454907, 23763a99124c40fa982e4ffa1c5fe29f; no voters:
I20260811 06:59:15.908507 29723 raft_consensus.cc:2454] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 118069c2bcab45d3b43d4ccd90454907 in term 0.
I20260811 06:59:15.908727 30103 raft_consensus.cc:2781] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:59:15.908864 30103 raft_consensus.cc:482] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:59:15.908986 30103 raft_consensus.cc:3037] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:15.911257 30103 raft_consensus.cc:504] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.911896 30103 leader_election.cc:288] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [CANDIDATE]: Term 1 election: Requested vote from peers 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3:36665), 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107)
I20260811 06:59:15.912333 29970 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f321453a5765401f9441982978d1d451" candidate_uuid: "118069c2bcab45d3b43d4ccd90454907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "23763a99124c40fa982e4ffa1c5fe29f"
I20260811 06:59:15.912468 29723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f321453a5765401f9441982978d1d451" candidate_uuid: "118069c2bcab45d3b43d4ccd90454907" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c800786a9d47adaba18767a63ec4a9"
I20260811 06:59:15.912542 29970 raft_consensus.cc:3037] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:15.912708 29723 raft_consensus.cc:3037] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:15.914773 29970 raft_consensus.cc:2454] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 118069c2bcab45d3b43d4ccd90454907 in term 1.
I20260811 06:59:15.914997 29723 raft_consensus.cc:2454] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 118069c2bcab45d3b43d4ccd90454907 in term 1.
I20260811 06:59:15.915192 29363 leader_election.cc:302] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 118069c2bcab45d3b43d4ccd90454907, 23763a99124c40fa982e4ffa1c5fe29f; no voters:
I20260811 06:59:15.915479 30103 raft_consensus.cc:2781] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:15.915954 30103 raft_consensus.cc:686] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 1 LEADER]: Becoming Leader. State: Replica: 118069c2bcab45d3b43d4ccd90454907, State: Running, Role: LEADER
I20260811 06:59:15.916514 30103 consensus_queue.cc:237] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } }
I20260811 06:59:15.921059 29205 catalog_manager.cc:5495] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 reported cstate change: term changed from 0 to 1, leader changed from <none> to 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1). New cstate: current_term: 1 leader_uuid: "118069c2bcab45d3b43d4ccd90454907" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } health_report { overall_health: UNKNOWN } } }
I20260811 06:59:15.960960 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:15.955727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.4.40.1:33191)
0811 06:59:15.955939 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:15.955944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:15.955961 (+ 17us) server_negotiation.cc:407] Connection header received
0811 06:59:15.956161 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:15.956165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:15.956243 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:15.956381 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:59:15.956719 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.957392 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.958931 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:15.959320 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:15.959409 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:15.959870 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:59:15.960379 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:59:15.960492 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:15.960586 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 06:59:15.960660 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 06:59:15.968943 29970 raft_consensus.cc:1261] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Refusing update from remote peer 118069c2bcab45d3b43d4ccd90454907: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:15.968945 29723 raft_consensus.cc:1261] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Refusing update from remote peer 118069c2bcab45d3b43d4ccd90454907: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:15.969882 30103 consensus_queue.cc:1035] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:15.970214 30110 consensus_queue.cc:1035] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [LEADER]: Connected to new peer: Peer: permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:15.996040 29204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:15.996373 29204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:15.999686 29204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Sending DeleteTablet for 3 replicas of tablet f321453a5765401f9441982978d1d451
I20260811 06:59:16.000826 29445 tablet_service.cc:1514] Processing DeleteTablet for tablet f321453a5765401f9441982978d1d451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:15 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59608
I20260811 06:59:16.000890 29930 tablet_service.cc:1514] Processing DeleteTablet for tablet f321453a5765401f9441982978d1d451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:15 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34428
I20260811 06:59:16.001293 29693 tablet_service.cc:1514] Processing DeleteTablet for tablet f321453a5765401f9441982978d1d451 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:15 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56478
I20260811 06:59:16.002477 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 29337
I20260811 06:59:16.016296 30130 tablet_replica.cc:330] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: stopping tablet replica
I20260811 06:59:16.016911 30130 raft_consensus.cc:2229] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:16.017594 30130 raft_consensus.cc:2258] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:16.019229 30129 tablet_replica.cc:330] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: stopping tablet replica
I20260811 06:59:16.019663 30129 raft_consensus.cc:2229] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:59:16.019923 30130 ts_tablet_manager.cc:1905] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:16.020365 30129 raft_consensus.cc:2258] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:16.022670 30129 ts_tablet_manager.cc:1905] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:16.026958 30130 ts_tablet_manager.cc:1918] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:16.027129 30130 log.cc:1196] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/f321453a5765401f9441982978d1d451
I20260811 06:59:16.027684 30130 ts_tablet_manager.cc:1939] T f321453a5765401f9441982978d1d451 P 23763a99124c40fa982e4ffa1c5fe29f: Deleting consensus metadata
I20260811 06:59:16.029546 29188 catalog_manager.cc:4841] TS 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3:36665): tablet f321453a5765401f9441982978d1d451 (table test-table [id=c9ff7ba784b1405980611d3ee86b65af]) successfully deleted
I20260811 06:59:16.029877 30129 ts_tablet_manager.cc:1918] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:16.030049 30129 log.cc:1196] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/f321453a5765401f9441982978d1d451
I20260811 06:59:16.030522 30129 ts_tablet_manager.cc:1939] T f321453a5765401f9441982978d1d451 P 118069c2bcab45d3b43d4ccd90454907: Deleting consensus metadata
I20260811 06:59:16.031939 30132 tablet_replica.cc:330] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: stopping tablet replica
I20260811 06:59:16.032449 30132 raft_consensus.cc:2229] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:16.032918 29189 catalog_manager.cc:4841] TS 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1:33191): tablet f321453a5765401f9441982978d1d451 (table test-table [id=c9ff7ba784b1405980611d3ee86b65af]) successfully deleted
I20260811 06:59:16.032950 30132 raft_consensus.cc:2258] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:16.035024 30132 ts_tablet_manager.cc:1905] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 06:59:16 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:59:16.041779 30132 ts_tablet_manager.cc:1918] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:16.041936 30132 log.cc:1196] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/f321453a5765401f9441982978d1d451
I20260811 06:59:16.042477 30132 ts_tablet_manager.cc:1939] T f321453a5765401f9441982978d1d451 P 88c800786a9d47adaba18767a63ec4a9: Deleting consensus metadata
I20260811 06:59:16.044250 29186 catalog_manager.cc:4841] TS 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107): tablet f321453a5765401f9441982978d1d451 (table test-table [id=c9ff7ba784b1405980611d3ee86b65af]) successfully deleted
I20260811 06:59:16.045913 30083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:16.003847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.4.40.1:33191)
0811 06:59:16.004049 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:16.004055 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:16.028536 (+ 24481us) server_negotiation.cc:407] Connection header received
0811 06:59:16.029114 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:16.029120 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:16.029221 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:16.029298 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:16.031473 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:16.032539 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:16.034375 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:16.034854 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:16.034990 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:16.039396 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:16.039406 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:16.039418 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:16.039453 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:16.042749 (+ 3296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:16.043471 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:16.043479 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:16.043482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:16.043637 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:16.044067 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:16.044072 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:16.044074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:16.044564 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:16.044763 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:16.045283 (+ 520us) server_negotiation.cc:299] Negotiation successful
0811 06:59:16.045572 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:59:16.046660 29557 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:16.146564 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 29337
I20260811 06:59:16.176754 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 29573
Aug 11 06:59:16 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:59:16.192160 30088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:16.177521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.4.40.2:38107)
0811 06:59:16.177719 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:16.177724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:16.177743 (+ 19us) server_negotiation.cc:407] Connection header received
0811 06:59:16.177888 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:16.177892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:16.177983 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:16.178065 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:16.179561 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:16.180385 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:16.181609 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:16.181967 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:16.182056 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:16.186259 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:16.186272 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:16.186287 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:16.186322 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:16.189176 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:16.189921 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:16.189934 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:16.189936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:16.190071 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:16.190497 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:16.190500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:16.190508 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:16.190875 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:16.191195 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:16.191499 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 06:59:16.191733 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:59:16.192843 29802 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:16.281980 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 29573
I20260811 06:59:16.309922 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 29818
Aug 11 06:59:16 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:59:16.329556 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:16.315136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.4.40.3:36665)
0811 06:59:16.315311 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:16.315316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:16.315337 (+ 21us) server_negotiation.cc:407] Connection header received
0811 06:59:16.315395 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:16.315399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:16.315473 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:16.315560 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:16.317029 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:16.317733 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:16.319067 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:16.319441 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:16.319533 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:16.323990 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:16.324013 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:16.324030 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:16.324063 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:16.326902 (+ 2839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:16.327562 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:16.327568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:16.327570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:16.327693 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:16.328062 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:16.328065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:16.328067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:16.328526 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:16.328743 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:16.329024 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 06:59:16.329184 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:59:16.330291 30045 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:16.423943 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 29818
I20260811 06:59:16.449184 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 29160
I20260811 06:59:16.450330 29275 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:16.559959 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 29160
I20260811 06:59:16.588919 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37531
--webserver_interface=127.4.40.62
--webserver_port=35803
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:37531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:16.941565 30145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:16.941900 30145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:16.942072 30145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:16.949223 30145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:16.949352 30145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:16.949455 30145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:16.949553 30145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:16.949630 30145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:16.959208 30145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:37531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=35803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:16.960487 30145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:16.961793 30145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:16.988055 30152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:16.988077 30154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:16.988035 30157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:16.989213 30145 server_base.cc:1034] running on GCE node
I20260811 06:59:16.990041 30145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:16.991559 30145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:16.992807 30145 hybrid_clock.cc:648] HybridClock initialized: now 1786431556992751 us; error 56 us; skew 500 ppm
I20260811 06:59:16.993199 30145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:16 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431556, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:16.996825 30145 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:16.998440 30145 webserver.cc:460] Webserver started at http://127.4.40.62:35803/ using document root <none> and password file <none>
I20260811 06:59:16.999199 30145 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:16.999368 30145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:17.005378 30145 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 06:59:17.008292 30166 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:17.009749 30145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:59:17.009958 30145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "561ef81780f04a80822de7800689a027"
format_stamp: "Formatted at 2026-08-11 06:59:13 on dist-test-slave-7vjv"
server_key: "964f28137fbabba0f7906170a1deaae0"
server_key_iv: "cd263cb78baa4bf020c00bdb31a35da6"
server_key_version: "encryptionkey@0"
I20260811 06:59:17.011039 30145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:17.030397 30145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:17.034935 30145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:17.035223 30145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:17.054067 30145 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:37531
I20260811 06:59:17.054111 30250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:37531 every 8 connection(s)
I20260811 06:59:17.055596 30145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:17.058414 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 30145
I20260811 06:59:17.059448 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:33191
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:37531
--webserver_port=39575
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:59:17.064394 30251 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:59:17.069345 30251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap starting.
I20260811 06:59:17.089897 30251 log.cc:824] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:17.102612 30251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:59:17.103242 30251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap complete.
I20260811 06:59:17.112512 30251 raft_consensus.cc:348] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:17.113348 30251 raft_consensus.cc:729] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561ef81780f04a80822de7800689a027, State: Initialized, Role: FOLLOWER
I20260811 06:59:17.113937 30251 consensus_queue.cc:260] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:17.114229 30251 raft_consensus.cc:388] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:59:17.114384 30251 raft_consensus.cc:482] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:59:17.114610 30251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 1 FOLLOWER]: Advancing to term 2
I20260811 06:59:17.118750 30251 raft_consensus.cc:504] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:17.119271 30251 leader_election.cc:302] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 561ef81780f04a80822de7800689a027; no voters:
I20260811 06:59:17.119944 30251 leader_election.cc:288] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 06:59:17.120086 30259 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 06:59:17.122526 30251 sys_catalog.cc:564] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:17.127280 30259 raft_consensus.cc:686] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 LEADER]: Becoming Leader. State: Replica: 561ef81780f04a80822de7800689a027, State: Running, Role: LEADER
I20260811 06:59:17.127905 30259 consensus_queue.cc:237] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:17.130822 30260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "561ef81780f04a80822de7800689a027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } } }
I20260811 06:59:17.131258 30260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:17.131884 30261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 561ef81780f04a80822de7800689a027. Latest consensus state: current_term: 2 leader_uuid: "561ef81780f04a80822de7800689a027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } } }
I20260811 06:59:17.132261 30261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:17.138546 30278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:17.146728 30278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:17.148126 30278 catalog_manager.cc:1259] Loaded cluster ID: c4550c75cf934703a5508e9d5130d86f
I20260811 06:59:17.148243 30278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:59:17.151525 30278 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 06:59:17.168426 30278 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 06:59:17.175403 30278 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:59:17.176707 30278 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Loaded TSK: 0
I20260811 06:59:17.177802 30278 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 06:59:17.437076 30256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:17.437345 30256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:17.437482 30256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:17.444598 30256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:17.444741 30256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:17.444934 30256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:59:17.454424 30256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:33191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=39575
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:17.455824 30256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:17.457201 30256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:17.472067 30296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:17.472627 30303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:17.472710 30297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:17.473282 30256 server_base.cc:1034] running on GCE node
I20260811 06:59:17.473941 30256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:17.474985 30256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:17.476217 30256 hybrid_clock.cc:648] HybridClock initialized: now 1786431557476157 us; error 51 us; skew 500 ppm
I20260811 06:59:17.476532 30256 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:17 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431557, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:17.479899 30256 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:59:17.481554 30256 webserver.cc:460] Webserver started at http://127.4.40.1:39575/ using document root <none> and password file <none>
I20260811 06:59:17.482260 30256 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:17.482414 30256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:17.487927 30256 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 06:59:17.490799 30318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:17.492286 30256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:59:17.492466 30256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "118069c2bcab45d3b43d4ccd90454907"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "0b3e2cfe39d9d9314305bab1272b3fb5"
server_key_iv: "9d89f57c5e57c11598c54b34dc48e4fe"
server_key_version: "encryptionkey@0"
I20260811 06:59:17.493449 30256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:17.511144 30256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:17.515810 30256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:17.516150 30256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:17.517211 30256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:17.518671 30256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:17.518784 30256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:17.518898 30256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:17.519003 30256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:17.556216 30256 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:33191
I20260811 06:59:17.556223 30514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:33191 every 8 connection(s)
I20260811 06:59:17.557614 30256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:59:17.561250 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 30256
I20260811 06:59:17.562000 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:38107
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:37531
--webserver_port=46445
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:59:17 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431557, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:17.581008 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:17.560423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:48249 (local address 127.4.40.62:37531)
0811 06:59:17.561098 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:17.561116 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:17.561659 (+ 543us) server_negotiation.cc:407] Connection header received
0811 06:59:17.563842 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:17.563853 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:17.564242 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:17.564580 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:17.566070 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:17.567263 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:17.568842 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:17.569452 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:17.569645 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:17.573144 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:17.573155 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:17.573174 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:17.573233 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:17.576626 (+ 3393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:17.577319 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:17.577332 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:17.577337 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:17.577489 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:17.577913 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:17.577918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:17.577921 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:17.578353 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:17.578558 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:17.579332 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 06:59:17.579742 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":285,"threads_started":1}
I20260811 06:59:17.581003 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:17.560588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:48249)
0811 06:59:17.561490 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:17.561530 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:17.563305 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:17.564740 (+ 1435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:17.564752 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:17.565345 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:17.565845 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:17.565862 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:17.567436 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:17.567440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:17.568696 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:17.568709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:17.569691 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:17.569698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:17.569987 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:17.570657 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:17.570679 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:17.572856 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:17.576763 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:17.576772 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:17.576785 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:17.577181 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:17.577602 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:17.577606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:17.577610 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:17.577797 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:17.578497 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:17.578505 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:17.578848 (+ 343us) client_negotiation.cc:769] Sending connection context
0811 06:59:17.579098 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 06:59:17.579657 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":341,"threads_started":1}
I20260811 06:59:17.583859 30515 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37531
I20260811 06:59:17.584216 30515 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:17.584949 30515 heartbeater.cc:511] Master 127.4.40.62:37531 requested a full tablet report, sending...
I20260811 06:59:17.586999 30191 ts_manager.cc:194] Registered new tserver with Master: 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1:33191)
I20260811 06:59:17.588521 30191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:48249
W20260811 06:59:17.918238 30530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:17.918534 30530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:17.918689 30530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:17.926138 30530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:17.926276 30530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:17.926436 30530 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:59:17.936161 30530 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:38107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=46445
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:17.937438 30530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:17.938750 30530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:17.951676 30538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:17.951764 30542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:17.951719 30540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:17.953310 30530 server_base.cc:1034] running on GCE node
I20260811 06:59:17.953907 30530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:17.954788 30530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:17.956013 30530 hybrid_clock.cc:648] HybridClock initialized: now 1786431557955979 us; error 43 us; skew 500 ppm
I20260811 06:59:17.956331 30530 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:17 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431557, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:17.959493 30530 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:59:17.960956 30530 webserver.cc:460] Webserver started at http://127.4.40.2:46445/ using document root <none> and password file <none>
I20260811 06:59:17.961637 30530 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:17.961795 30530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:17.967113 30530 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 06:59:17.969839 30556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:17.971101 30530 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:59:17.971271 30530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "88c800786a9d47adaba18767a63ec4a9"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "96eb947b354ef77abf6a2ab595279187"
server_key_iv: "695f1e47a60cca2404f1841f82b8f2a2"
server_key_version: "encryptionkey@0"
I20260811 06:59:17.972281 30530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:17.995853 30530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:18.000284 30530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:18.000561 30530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:18.001529 30530 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:18.003063 30530 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:18.003183 30530 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:18.003306 30530 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:18.003392 30530 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:18.037408 30530 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:38107
I20260811 06:59:18.037528 30744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:38107 every 8 connection(s)
I20260811 06:59:18.038870 30530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:59:18.043555 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 30530
I20260811 06:59:18.044275 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:36665
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:37531
--webserver_port=38269
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 06:59:18 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431557, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:18.061604 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.041481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:34835 (local address 127.4.40.62:37531)
0811 06:59:18.041650 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.041654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.042713 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 06:59:18.044739 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.044745 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.044831 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.044939 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:18.046525 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.047633 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.049855 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.050627 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.050775 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.054828 (+ 4053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:18.054834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:18.054837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:18.054870 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:18.058302 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:18.059207 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:18.059217 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:18.059220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:18.059344 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:18.059833 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:18.059836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:18.059838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:18.060159 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:18.060325 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.061009 (+ 684us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.061241 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:59:18.062661 30747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.041915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:34835)
0811 06:59:18.042558 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.042604 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.044462 (+ 1858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.045182 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.045189 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.045674 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:18.046336 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.046358 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.047793 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.047800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.049693 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.049710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.050849 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.050856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.051212 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.051972 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:18.052009 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:18.054625 (+ 2616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:18.058486 (+ 3861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:18.058495 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:18.058510 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:18.059065 (+ 555us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:18.059478 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:18.059483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:18.059487 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:18.059725 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:18.060304 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:18.060312 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:18.060619 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.060885 (+ 266us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.061391 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":253,"threads_started":1}
I20260811 06:59:18.064113 30745 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37531
I20260811 06:59:18.064424 30745 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:18.065124 30745 heartbeater.cc:511] Master 127.4.40.62:37531 requested a full tablet report, sending...
I20260811 06:59:18.066622 30191 ts_manager.cc:194] Registered new tserver with Master: 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107)
I20260811 06:59:18.067494 30191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:34835
W20260811 06:59:18.392745 30749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:18.392994 30749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:18.393085 30749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:18.400254 30749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:18.400357 30749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:18.400489 30749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:59:18.411842 30749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:36665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=38269
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:18.413168 30749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:18.414599 30749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:18.429287 30758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:18.429240 30756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:18.429344 30754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:18.430585 30749 server_base.cc:1034] running on GCE node
I20260811 06:59:18.431212 30749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:18.432176 30749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:18.433396 30749 hybrid_clock.cc:648] HybridClock initialized: now 1786431558433356 us; error 48 us; skew 500 ppm
I20260811 06:59:18.433704 30749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:18 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:18.436924 30749 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:59:18.438270 30749 webserver.cc:460] Webserver started at http://127.4.40.3:38269/ using document root <none> and password file <none>
I20260811 06:59:18.438902 30749 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:18.439013 30749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:18.444298 30749 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 06:59:18.446806 30766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:18.448017 30749 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 06:59:18.448215 30749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "23763a99124c40fa982e4ffa1c5fe29f"
format_stamp: "Formatted at 2026-08-11 06:59:15 on dist-test-slave-7vjv"
server_key: "3ec70d95a1a04908ede8305dea97b2ab"
server_key_iv: "650213d819f8e6e275c2b871480772cd"
server_key_version: "encryptionkey@0"
I20260811 06:59:18.449177 30749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:18.464808 30749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:18.468928 30749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:18.469197 30749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:18.470163 30749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:18.471415 30749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:18.471544 30749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:18.471659 30749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:18.471747 30749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:18.505618 30749 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:36665
I20260811 06:59:18.505677 30962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:36665 every 8 connection(s)
I20260811 06:59:18.507091 30749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:59:18.511024 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 30749
Aug 11 06:59:18 dist-test-slave-7vjv krb5kdc[29134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431558, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:18.529402 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.511887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48358 (local address 127.4.40.62:37531)
0811 06:59:18.512445 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.512450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.512773 (+ 323us) server_negotiation.cc:407] Connection header received
0811 06:59:18.512943 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.512947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.513011 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.513125 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:18.515215 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.516483 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.518531 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.519141 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.519271 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.522861 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:18.522868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:18.522872 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:18.522913 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:18.526317 (+ 3404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:18.527202 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:18.527206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:18.527208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:18.527330 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:18.527716 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:18.527722 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:18.527725 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:18.528066 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:18.528342 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.528709 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.528943 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"spinlock_wait_cycles":14720,"thread_start_us":282,"threads_started":1}
I20260811 06:59:18.530853 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.509529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:48471 (local address 127.4.40.62:37531)
0811 06:59:18.509721 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.509725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.510834 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 06:59:18.513170 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.513177 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.513245 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.513353 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:18.515446 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.516532 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.518812 (+ 2280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.519500 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.519596 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.523888 (+ 4292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:18.523895 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:18.523900 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:18.523953 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:18.527331 (+ 3378us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:18.528225 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:18.528230 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:18.528233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:18.528398 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:18.528927 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:18.528934 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:18.528939 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:18.529318 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:18.529436 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.530358 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.530516 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:59:18.532241 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.509981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:48471)
0811 06:59:18.510682 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.510726 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.512904 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.513629 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.513639 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.514411 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:18.515251 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.515271 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.516669 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.516674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.518674 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.518686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.519740 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.519751 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.519996 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.520673 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:18.520709 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:18.523680 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:18.527501 (+ 3821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:18.527509 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:18.527521 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:18.528127 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:18.528517 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:18.528520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:18.528522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:18.528787 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:18.529463 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:18.529472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:18.529909 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.530261 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.530804 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":284,"threads_started":1}
I20260811 06:59:18.533524 30963 heartbeater.cc:347] Connected to a master server at 127.4.40.62:37531
I20260811 06:59:18.533803 30963 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:18.534495 30963 heartbeater.cc:511] Master 127.4.40.62:37531 requested a full tablet report, sending...
I20260811 06:59:18.535854 30191 ts_manager.cc:194] Registered new tserver with Master: 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3:36665)
I20260811 06:59:18.536720 30191 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:48471
I20260811 06:59:18.544377 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:59:18.550962 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.545284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48374 (local address 127.4.40.62:37531)
0811 06:59:18.545446 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.545451 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.546006 (+ 555us) server_negotiation.cc:407] Connection header received
0811 06:59:18.546181 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.546184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.546323 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.546441 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:59:18.546933 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.547673 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.549135 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.549554 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.549643 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.550069 (+ 426us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:59:18.550540 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:59:18.550640 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.550728 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.550787 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 06:59:18.551779 30191 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:59:18.560204 30191 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:59:18.591558 30515 heartbeater.cc:503] Master 127.4.40.62:37531 was elected leader, sending a full tablet report...
I20260811 06:59:18.598999 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.572424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.4.40.2:38107)
0811 06:59:18.590152 (+ 17728us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.590159 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.590182 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:59:18.590257 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.590262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.590682 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.591078 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:18.591884 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.593540 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.596781 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.597888 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.598029 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.598241 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.598512 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.598673 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17514,"thread_start_us":17395,"threads_started":1}
I20260811 06:59:18.599771 30990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.571830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:38107 (local address 127.0.0.1:56508)
0811 06:59:18.589438 (+ 17608us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.589462 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.589596 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.590907 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.590914 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.590955 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:18.591435 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.591447 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.593680 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.593685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.596615 (+ 2930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.596627 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.598080 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.598086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.598258 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.598319 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.598453 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.598622 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17463,"mutex_wait_us":111,"spinlock_wait_cycles":384,"thread_start_us":17717,"threads_started":1}
I20260811 06:59:18.599900 30985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.572559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:36665 (local address 127.0.0.1:34446)
0811 06:59:18.589819 (+ 17260us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.589836 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.589948 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.590920 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.590927 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.590961 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:18.592260 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.592274 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.594024 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.594053 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.596444 (+ 2391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.596458 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.599025 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.599031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.599178 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.599207 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.599280 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.599407 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17114,"mutex_wait_us":226,"spinlock_wait_cycles":6784,"thread_start_us":16590,"threads_started":1}
I20260811 06:59:18.600116 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.571554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59614)
0811 06:59:18.587185 (+ 15631us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.587245 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.587451 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.591492 (+ 4041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.591497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.591527 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:18.592088 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.592130 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.593910 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.593915 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.596765 (+ 2850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.596777 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.598912 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.598917 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.599300 (+ 383us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.599327 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.599393 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.599489 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15306,"mutex_wait_us":139,"thread_start_us":15134,"threads_started":1}
I20260811 06:59:18.600528 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.571783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59614 (local address 127.4.40.1:33191)
0811 06:59:18.590587 (+ 18804us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.590594 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.590619 (+ 25us) server_negotiation.cc:407] Connection header received
0811 06:59:18.590718 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.590724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.590920 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.591111 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:18.592338 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.593776 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.596959 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.598371 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.598496 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.598702 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.599480 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.599664 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":18628,"thread_start_us":18483,"threads_started":1}
I20260811 06:59:18.600690 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.573261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.4.40.3:36665)
0811 06:59:18.588918 (+ 15657us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.588930 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.590073 (+ 1143us) server_negotiation.cc:407] Connection header received
0811 06:59:18.590191 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.590200 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.590441 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.590649 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:18.592503 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.593883 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.596625 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.598051 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.598176 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.598394 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.600168 (+ 1774us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.600349 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15466,"thread_start_us":15319,"threads_started":1}
I20260811 06:59:18.603866 30851 tablet_service.cc:1467] Processing CreateTablet for tablet 3f333296c893496db62cccdf354b879d (DEFAULT_TABLE table=test-table [id=de03f482f0104d1e9791e00e546e9416]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:18.603863 30405 tablet_service.cc:1467] Processing CreateTablet for tablet 3f333296c893496db62cccdf354b879d (DEFAULT_TABLE table=test-table [id=de03f482f0104d1e9791e00e546e9416]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:18.603864 30635 tablet_service.cc:1467] Processing CreateTablet for tablet 3f333296c893496db62cccdf354b879d (DEFAULT_TABLE table=test-table [id=de03f482f0104d1e9791e00e546e9416]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:18.605208 30851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f333296c893496db62cccdf354b879d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:18.605257 30635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f333296c893496db62cccdf354b879d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:18.605275 30405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f333296c893496db62cccdf354b879d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:18.617535 30992 tablet_bootstrap.cc:492] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Bootstrap starting.
I20260811 06:59:18.620230 30998 tablet_bootstrap.cc:492] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Bootstrap starting.
I20260811 06:59:18.621809 30992 tablet_bootstrap.cc:654] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:18.623288 30992 log.cc:824] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:18.623811 30998 tablet_bootstrap.cc:654] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:18.624886 30997 tablet_bootstrap.cc:492] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Bootstrap starting.
I20260811 06:59:18.625207 30998 log.cc:824] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:18.627946 30998 tablet_bootstrap.cc:492] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: No bootstrap required, opened a new log
I20260811 06:59:18.628014 30992 tablet_bootstrap.cc:492] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: No bootstrap required, opened a new log
I20260811 06:59:18.628253 30998 ts_tablet_manager.cc:1397] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Time spent bootstrapping tablet: real 0.008s user 0.008s sys 0.000s
I20260811 06:59:18.628335 30992 ts_tablet_manager.cc:1397] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260811 06:59:18.628463 30997 tablet_bootstrap.cc:654] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:18.629683 30997 log.cc:824] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:18.634670 30997 tablet_bootstrap.cc:492] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: No bootstrap required, opened a new log
I20260811 06:59:18.634930 30997 ts_tablet_manager.cc:1397] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 06:59:18.635944 30998 raft_consensus.cc:348] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.636381 30998 raft_consensus.cc:374] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:18.636475 30998 raft_consensus.cc:729] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c800786a9d47adaba18767a63ec4a9, State: Initialized, Role: FOLLOWER
I20260811 06:59:18.637094 30998 consensus_queue.cc:260] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.639017 30992 raft_consensus.cc:348] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.639389 30992 raft_consensus.cc:374] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:18.639477 30992 raft_consensus.cc:729] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23763a99124c40fa982e4ffa1c5fe29f, State: Initialized, Role: FOLLOWER
I20260811 06:59:18.640062 30992 consensus_queue.cc:260] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.644263 30745 heartbeater.cc:503] Master 127.4.40.62:37531 was elected leader, sending a full tablet report...
I20260811 06:59:18.644888 30998 ts_tablet_manager.cc:1428] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Time spent starting tablet: real 0.017s user 0.008s sys 0.008s
I20260811 06:59:18.645473 30992 ts_tablet_manager.cc:1428] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
I20260811 06:59:18.645809 30963 heartbeater.cc:503] Master 127.4.40.62:37531 was elected leader, sending a full tablet report...
I20260811 06:59:18.646072 30997 raft_consensus.cc:348] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.646517 30997 raft_consensus.cc:374] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:18.646622 30997 raft_consensus.cc:729] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 118069c2bcab45d3b43d4ccd90454907, State: Initialized, Role: FOLLOWER
I20260811 06:59:18.647684 30997 consensus_queue.cc:260] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.649523 30997 ts_tablet_manager.cc:1428] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260811 06:59:18.758746 30968 tablet.cc:2369] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:59:18.790968 30746 tablet.cc:2369] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 06:59:18.809902 30516 tablet.cc:2369] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:18.897218 31007 raft_consensus.cc:482] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:59:18.897555 31007 raft_consensus.cc:504] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.899345 31007 leader_election.cc:288] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1:33191), 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107)
I20260811 06:59:18.905788 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.899919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:33623 (local address 127.4.40.2:38107)
0811 06:59:18.900155 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.900161 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.900183 (+ 22us) server_negotiation.cc:407] Connection header received
0811 06:59:18.900382 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.900390 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.900496 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.900613 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:18.901063 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.902029 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.903841 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.904924 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.905026 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.905096 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.905446 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.905551 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:59:18.905908 30969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.899674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:38107 (local address 127.4.40.3:33623)
0811 06:59:18.899988 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.900010 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.900189 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.900640 (+ 451us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.900644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.900668 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:18.900903 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.900917 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.902158 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.902163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.903678 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.903690 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.905085 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.905091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.905283 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.905323 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.905426 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.905580 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"mutex_wait_us":1}
I20260811 06:59:18.906728 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.899919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:52721 (local address 127.4.40.1:33191)
0811 06:59:18.900152 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.900157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.900827 (+ 670us) server_negotiation.cc:407] Connection header received
0811 06:59:18.900984 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.900991 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.901085 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.901153 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:18.901731 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.902641 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.904609 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.905612 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.905749 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.905843 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.906406 (+ 563us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.906543 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 06:59:18.906759 31010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.899732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.4.40.3:52721)
0811 06:59:18.900705 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:18.900725 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:18.900851 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:18.901203 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:18.901208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:18.901232 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:18.901583 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.901594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.902808 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.902812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.904440 (+ 1628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:18.904456 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.905937 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.905948 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:18.906239 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.906264 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 06:59:18.906342 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 06:59:18.906424 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"thread_start_us":618,"threads_started":1}
I20260811 06:59:18.907080 30666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f333296c893496db62cccdf354b879d" candidate_uuid: "23763a99124c40fa982e4ffa1c5fe29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c800786a9d47adaba18767a63ec4a9" is_pre_election: true
I20260811 06:59:18.907594 30666 raft_consensus.cc:2454] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23763a99124c40fa982e4ffa1c5fe29f in term 0.
I20260811 06:59:18.907716 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f333296c893496db62cccdf354b879d" candidate_uuid: "23763a99124c40fa982e4ffa1c5fe29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118069c2bcab45d3b43d4ccd90454907" is_pre_election: true
I20260811 06:59:18.908284 30769 leader_election.cc:302] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23763a99124c40fa982e4ffa1c5fe29f, 88c800786a9d47adaba18767a63ec4a9; no voters:
I20260811 06:59:18.908294 30432 raft_consensus.cc:2454] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23763a99124c40fa982e4ffa1c5fe29f in term 0.
I20260811 06:59:18.908672 31007 raft_consensus.cc:2781] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:59:18.908854 31007 raft_consensus.cc:482] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:59:18.908970 31007 raft_consensus.cc:3037] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:18.911192 31007 raft_consensus.cc:504] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.911816 31007 leader_election.cc:288] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [CANDIDATE]: Term 1 election: Requested vote from peers 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1:33191), 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107)
I20260811 06:59:18.912218 30432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f333296c893496db62cccdf354b879d" candidate_uuid: "23763a99124c40fa982e4ffa1c5fe29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "118069c2bcab45d3b43d4ccd90454907"
I20260811 06:59:18.912395 30666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f333296c893496db62cccdf354b879d" candidate_uuid: "23763a99124c40fa982e4ffa1c5fe29f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c800786a9d47adaba18767a63ec4a9"
I20260811 06:59:18.912580 30432 raft_consensus.cc:3037] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:18.912599 30666 raft_consensus.cc:3037] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:18.914940 30666 raft_consensus.cc:2454] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23763a99124c40fa982e4ffa1c5fe29f in term 1.
I20260811 06:59:18.915148 30432 raft_consensus.cc:2454] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23763a99124c40fa982e4ffa1c5fe29f in term 1.
I20260811 06:59:18.915340 30769 leader_election.cc:302] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23763a99124c40fa982e4ffa1c5fe29f, 88c800786a9d47adaba18767a63ec4a9; no voters:
I20260811 06:59:18.915671 31007 raft_consensus.cc:2781] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:18.916303 31007 raft_consensus.cc:686] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 1 LEADER]: Becoming Leader. State: Replica: 23763a99124c40fa982e4ffa1c5fe29f, State: Running, Role: LEADER
I20260811 06:59:18.916781 31007 consensus_queue.cc:237] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } }
I20260811 06:59:18.921540 30191 catalog_manager.cc:5495] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f reported cstate change: term changed from 0 to 1, leader changed from <none> to 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3). New cstate: current_term: 1 leader_uuid: "23763a99124c40fa982e4ffa1c5fe29f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23763a99124c40fa982e4ffa1c5fe29f" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36665 } health_report { overall_health: HEALTHY } } }
I20260811 06:59:18.966398 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:18.960966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.4.40.3:36665)
0811 06:59:18.961151 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:18.961157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:18.961323 (+ 166us) server_negotiation.cc:407] Connection header received
0811 06:59:18.961548 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:18.961552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:18.961625 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:18.961775 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:59:18.962144 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.962859 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.964285 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:18.964711 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:18.964846 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:18.965332 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:59:18.965809 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:59:18.965977 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:18.966045 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 06:59:18.966122 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 06:59:18.974260 30666 raft_consensus.cc:1261] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Refusing update from remote peer 23763a99124c40fa982e4ffa1c5fe29f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:18.974260 30432 raft_consensus.cc:1261] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Refusing update from remote peer 23763a99124c40fa982e4ffa1c5fe29f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:18.975374 31007 consensus_queue.cc:1035] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [LEADER]: Connected to new peer: Peer: permanent_uuid: "118069c2bcab45d3b43d4ccd90454907" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 33191 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:18.975764 31011 consensus_queue.cc:1035] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88c800786a9d47adaba18767a63ec4a9" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 38107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:18.987993 31016 mvcc.cc:205] Tried to move back new op lower bound from 7317223665547812864 to 7317223665327378432. Current Snapshot: MvccSnapshot[applied={T|T < 7317223665547812864}]
I20260811 06:59:18.999636 30191 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:18.999887 30191 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:19.003371 30191 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Sending DeleteTablet for 3 replicas of tablet 3f333296c893496db62cccdf354b879d
I20260811 06:59:19.004684 30405 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f333296c893496db62cccdf354b879d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:18 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:59614
I20260811 06:59:19.004685 30635 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f333296c893496db62cccdf354b879d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:18 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:56508
I20260811 06:59:19.005265 30851 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f333296c893496db62cccdf354b879d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:18 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34446
I20260811 06:59:19.006130 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 30256
I20260811 06:59:19.018549 31033 tablet_replica.cc:330] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: stopping tablet replica
I20260811 06:59:19.018850 31032 tablet_replica.cc:330] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: stopping tablet replica
I20260811 06:59:19.019178 31033 raft_consensus.cc:2229] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:19.019466 31032 raft_consensus.cc:2229] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:19.019816 31033 raft_consensus.cc:2258] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:19.020126 31032 raft_consensus.cc:2258] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:19.022395 31032 ts_tablet_manager.cc:1905] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:19.022387 31033 ts_tablet_manager.cc:1905] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:19.022781 31035 tablet_replica.cc:330] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: stopping tablet replica
I20260811 06:59:19.024056 31035 raft_consensus.cc:2229] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:59:19.024762 31035 raft_consensus.cc:2258] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:19.028196 30984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.006977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59626 (local address 127.4.40.1:33191)
0811 06:59:19.007149 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:19.007155 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:19.007234 (+ 79us) server_negotiation.cc:407] Connection header received
0811 06:59:19.007435 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:19.007440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:19.007516 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:19.007629 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:19.009754 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:19.010927 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:19.013515 (+ 2588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:19.014191 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:19.014325 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:19.018557 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:19.018575 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:19.018592 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:19.018635 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:19.022924 (+ 4289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:19.025419 (+ 2495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:19.025427 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:19.025430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:19.025626 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:19.026044 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:19.026049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:19.026051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:19.026591 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:19.026918 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:19.027244 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 06:59:19.027498 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 06:59:19.028544 31035 ts_tablet_manager.cc:1905] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:19.028889 30506 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:19.030241 31033 ts_tablet_manager.cc:1918] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:19.030409 31033 log.cc:1196] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/3f333296c893496db62cccdf354b879d
I20260811 06:59:19.031075 31033 ts_tablet_manager.cc:1939] T 3f333296c893496db62cccdf354b879d P 88c800786a9d47adaba18767a63ec4a9: Deleting consensus metadata
I20260811 06:59:19.033371 30167 catalog_manager.cc:4841] TS 88c800786a9d47adaba18767a63ec4a9 (127.4.40.2:38107): tablet 3f333296c893496db62cccdf354b879d (table test-table [id=de03f482f0104d1e9791e00e546e9416]) successfully deleted
I20260811 06:59:19.036211 31035 ts_tablet_manager.cc:1918] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:19.036386 31035 log.cc:1196] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/3f333296c893496db62cccdf354b879d
I20260811 06:59:19.037001 31035 ts_tablet_manager.cc:1939] T 3f333296c893496db62cccdf354b879d P 23763a99124c40fa982e4ffa1c5fe29f: Deleting consensus metadata
I20260811 06:59:19.038782 30171 catalog_manager.cc:4841] TS 23763a99124c40fa982e4ffa1c5fe29f (127.4.40.3:36665): tablet 3f333296c893496db62cccdf354b879d (table test-table [id=de03f482f0104d1e9791e00e546e9416]) successfully deleted
I20260811 06:59:19.186453 31032 ts_tablet_manager.cc:1918] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:19.186707 31032 log.cc:1196] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/3f333296c893496db62cccdf354b879d
I20260811 06:59:19.186712 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 30256
I20260811 06:59:19.212630 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 30530
W20260811 06:59:19.212553 30172 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:33191 (error 108)
I20260811 06:59:19.216836 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.214061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59634)
0811 06:59:19.216479 (+ 2418us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.216612 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2270,"thread_start_us":2160,"threads_started":1}
W20260811 06:59:19.217118 30172 catalog_manager.cc:4584] TS 118069c2bcab45d3b43d4ccd90454907 (127.4.40.1:33191): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f333296c893496db62cccdf354b879d: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
I20260811 06:59:19.228224 30988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.213520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56522 (local address 127.4.40.2:38107)
0811 06:59:19.213731 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:19.213738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:19.213846 (+ 108us) server_negotiation.cc:407] Connection header received
0811 06:59:19.214067 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:19.214073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:19.214201 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:19.214380 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:19.216216 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:19.217066 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:19.218522 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:19.218990 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:19.219132 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:19.221992 (+ 2860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:19.222014 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:19.222028 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:19.222061 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:19.225202 (+ 3141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:19.226071 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:19.226078 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:19.226081 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:19.226255 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:19.226684 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:19.226687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:19.226689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:19.227130 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:19.227349 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:19.227730 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 06:59:19.227899 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 06:59:19.229197 30739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:19.270803 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.270236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59638)
0811 06:59:19.270491 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.270621 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 06:59:19.271965 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.271535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59648)
0811 06:59:19.271707 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.271792 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 06:59:19.320263 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 30530
I20260811 06:59:19.332742 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.332302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59660)
0811 06:59:19.332492 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.332586 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 06:59:19.333813 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.333446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59674)
0811 06:59:19.333609 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.333677 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:59:19.356021 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 30749
I20260811 06:59:19.370736 30991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.357042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34462 (local address 127.4.40.3:36665)
0811 06:59:19.357276 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:19.357282 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:19.357310 (+ 28us) server_negotiation.cc:407] Connection header received
0811 06:59:19.357374 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:19.357378 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:19.357436 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:19.357546 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:19.358888 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:19.359757 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:19.361196 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:19.361624 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:19.361758 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:19.364449 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:19.364461 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:19.364476 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:19.364518 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:19.367519 (+ 3001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:19.368379 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:19.368387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:19.368389 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:19.368549 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:19.368970 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:19.368974 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:19.368976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:19.369445 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:19.369820 (+ 375us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:19.370142 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 06:59:19.370371 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 06:59:19.371699 30950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:19.415259 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.414630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59688)
0811 06:59:19.414884 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.415011 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 06:59:19.416531 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.415981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:59700)
0811 06:59:19.416247 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.416353 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 06:59:19.498852 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 30749
I20260811 06:59:19.529601 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 30145
I20260811 06:59:19.530758 30242 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:19.646651 31038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:19.646264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:33191 (local address 127.0.0.1:32828)
0811 06:59:19.646424 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:19.646507 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:33191: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 06:59:19.646692 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 30145
I20260811 06:59:19.677796 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37531
--webserver_interface=127.4.40.62
--webserver_port=35803
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:37531
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:20.020685 31049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:20.021032 31049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:20.021180 31049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:20.028286 31049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:20.028422 31049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:20.028524 31049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:20.028620 31049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:20.028692 31049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:20.038422 31049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:37531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=35803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:20.039851 31049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:20.041237 31049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:20.052881 31057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:20.052997 31055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:20.053412 31062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:20.053741 31049 server_base.cc:1034] running on GCE node
I20260811 06:59:20.054499 31049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:20.055881 31049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:20.057085 31049 hybrid_clock.cc:648] HybridClock initialized: now 1786431560057034 us; error 48 us; skew 500 ppm
I20260811 06:59:20.057427 31049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:20 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:20.066337 31049 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:20.067886 31049 webserver.cc:460] Webserver started at http://127.4.40.62:35803/ using document root <none> and password file <none>
I20260811 06:59:20.068627 31049 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:20.068764 31049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:20.074010 31049 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 06:59:20.076818 31072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:20.078079 31049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:59:20.078253 31049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "561ef81780f04a80822de7800689a027"
format_stamp: "Formatted at 2026-08-11 06:59:13 on dist-test-slave-7vjv"
server_key: "964f28137fbabba0f7906170a1deaae0"
server_key_iv: "cd263cb78baa4bf020c00bdb31a35da6"
server_key_version: "encryptionkey@0"
I20260811 06:59:20.079211 31049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:20.119866 31049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:20.124423 31049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:20.124744 31049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:20.144935 31049 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:37531
I20260811 06:59:20.144979 31162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:37531 every 8 connection(s)
I20260811 06:59:20.146569 31049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:20.150740 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 31049
I20260811 06:59:20.151556 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:33191
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:37531
--webserver_port=39575
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 06:59:20.153631 31166 sys_catalog.cc:263] Verifying existing consensus state
I20260811 06:59:20.156919 31166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap starting.
I20260811 06:59:20.177523 31166 log.cc:824] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:20.193849 31166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 06:59:20.194387 31166 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027: Bootstrap complete.
I20260811 06:59:20.203567 31166 raft_consensus.cc:348] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:20.204566 31166 raft_consensus.cc:729] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 561ef81780f04a80822de7800689a027, State: Initialized, Role: FOLLOWER
I20260811 06:59:20.205214 31166 consensus_queue.cc:260] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:20.205457 31166 raft_consensus.cc:388] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:59:20.205593 31166 raft_consensus.cc:482] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:59:20.205775 31166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 2 FOLLOWER]: Advancing to term 3
I20260811 06:59:20.209686 31166 raft_consensus.cc:504] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:20.210191 31166 leader_election.cc:302] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 561ef81780f04a80822de7800689a027; no voters:
I20260811 06:59:20.210901 31166 leader_election.cc:288] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 06:59:20.211077 31171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 06:59:20.212229 31171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [term 3 LEADER]: Becoming Leader. State: Replica: 561ef81780f04a80822de7800689a027, State: Running, Role: LEADER
I20260811 06:59:20.212894 31171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } }
I20260811 06:59:20.213277 31166 sys_catalog.cc:564] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:20.215667 31173 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 561ef81780f04a80822de7800689a027. Latest consensus state: current_term: 3 leader_uuid: "561ef81780f04a80822de7800689a027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } } }
I20260811 06:59:20.216145 31173 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:20.217582 31172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "561ef81780f04a80822de7800689a027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "561ef81780f04a80822de7800689a027" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37531 } } }
I20260811 06:59:20.217854 31172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 561ef81780f04a80822de7800689a027 [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:20.218659 31181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:20.227900 31181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:20.229497 31181 catalog_manager.cc:1259] Loaded cluster ID: c4550c75cf934703a5508e9d5130d86f
I20260811 06:59:20.229638 31181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 06:59:20.231775 31181 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f189543a9ad google::LogMessage::Fail() at ??:0
@ 0x7f189543d88a google::LogMessage::SendToLog() at ??:0
@ 0x7f189543a4de google::LogMessage::Flush() at ??:0
@ 0x7f189543ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f18aae6d512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f18aae6a976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f18aaef2610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f18aaef2439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1898086345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f189808991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1898089799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1898064eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f189b33e6db start_thread at ??:0
@ 0x7f18942d461f clone at ??:0
W20260811 06:59:20.603749 31169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:20.604051 31169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:20.604245 31169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:20.611928 31169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:20.612061 31169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:20.612283 31169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:59:20.622218 31169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:33191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=39575
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:20.623469 31169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:20.624925 31169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:20.639099 31243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:20.639150 31236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:20.639195 31234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:20.641572 31169 server_base.cc:1034] running on GCE node
I20260811 06:59:20.642244 31169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:20.643182 31169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:20.644397 31169 hybrid_clock.cc:648] HybridClock initialized: now 1786431560644345 us; error 56 us; skew 500 ppm
I20260811 06:59:20.644704 31169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:20 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431560, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:20.647794 31169 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:59:20.649299 31169 webserver.cc:460] Webserver started at http://127.4.40.1:39575/ using document root <none> and password file <none>
I20260811 06:59:20.650005 31169 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:20.650144 31169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:20.655243 31169 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:59:20.658036 31259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:20.659230 31169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 06:59:20.659403 31169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "118069c2bcab45d3b43d4ccd90454907"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "0b3e2cfe39d9d9314305bab1272b3fb5"
server_key_iv: "9d89f57c5e57c11598c54b34dc48e4fe"
server_key_version: "encryptionkey@0"
I20260811 06:59:20.660419 31169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:20.671872 31169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:20.676676 31169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:20.677083 31169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:20.678463 31169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:20.684001 31268 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
W20260811 06:59:20.689378 31268 ts_tablet_manager.cc:1852] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 06:59:20.690325 31268 ts_tablet_manager.cc:1905] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:20.697470 31268 ts_tablet_manager.cc:1918] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:20.697732 31268 log.cc:1196] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/3f333296c893496db62cccdf354b879d
I20260811 06:59:20.698187 31268 ts_tablet_manager.cc:1939] T 3f333296c893496db62cccdf354b879d P 118069c2bcab45d3b43d4ccd90454907: Deleting consensus metadata
I20260811 06:59:20.698604 31169 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 06:59:20.698733 31169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.018s user 0.004s sys 0.000s
I20260811 06:59:20.698871 31169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:20.698969 31169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:20.733556 31169 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:33191
I20260811 06:59:20.733630 31456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:33191 every 8 connection(s)
I20260811 06:59:20.734884 31169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:59:20.739471 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:20.737757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:37159)
0811 06:59:20.738489 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:20.738662 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":308,"threads_started":1}
W20260811 06:59:20.740267 31457 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:37531 (0 consecutive failures): Network error: Failed to ping master at 127.4.40.62:37531: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
I20260811 06:59:20.741032 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:20.740712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:46675)
0811 06:59:20.740879 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:20.740939 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 06:59:20.741899 31460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:20.741587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:40451)
0811 06:59:20.741714 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:20.741784 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 06:59:20.742112 31457 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:59:20.743985 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 31169
I20260811 06:59:20.744659 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:38107
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:37531
--webserver_port=46445
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:21.081971 31463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:21.082315 31463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:21.082494 31463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:21.089495 31463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:21.089797 31463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:21.090031 31463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:59:21.100333 31463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:38107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=46445
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:21.101593 31463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:21.102990 31463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:21.115967 31474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:21.116042 31472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:21.116015 31471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:21.117336 31463 server_base.cc:1034] running on GCE node
I20260811 06:59:21.117955 31463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:21.118848 31463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:21.120062 31463 hybrid_clock.cc:648] HybridClock initialized: now 1786431561120029 us; error 59 us; skew 500 ppm
I20260811 06:59:21.120400 31463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:21 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431561, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:21.123436 31463 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:59:21.124816 31463 webserver.cc:460] Webserver started at http://127.4.40.2:46445/ using document root <none> and password file <none>
I20260811 06:59:21.125489 31463 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:21.125633 31463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:21.130935 31463 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 06:59:21.133669 31488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:21.134938 31463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 06:59:21.135075 31463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "88c800786a9d47adaba18767a63ec4a9"
format_stamp: "Formatted at 2026-08-11 06:59:14 on dist-test-slave-7vjv"
server_key: "96eb947b354ef77abf6a2ab595279187"
server_key_iv: "695f1e47a60cca2404f1841f82b8f2a2"
server_key_version: "encryptionkey@0"
I20260811 06:59:21.136004 31463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:21.163755 31463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:21.167764 31463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:21.168005 31463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:21.169072 31463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:21.170351 31463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:21.170446 31463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:21.170526 31463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:21.170581 31463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:21.204841 31463 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:38107
I20260811 06:59:21.204927 31682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:38107 every 8 connection(s)
I20260811 06:59:21.206280 31463 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:59:21.210908 31689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.209065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:48547)
0811 06:59:21.209827 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.210040 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":295,"threads_started":1}
W20260811 06:59:21.211565 31686 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:37531 (0 consecutive failures): Network error: Failed to ping master at 127.4.40.62:37531: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
I20260811 06:59:21.212476 31689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.212140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:39123)
0811 06:59:21.212292 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.212367 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 06:59:21.213311 31689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.212972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:36397)
0811 06:59:21.213119 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.213196 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 06:59:21.213587 31686 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:59:21.214385 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 31463
I20260811 06:59:21.215010 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:36665
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:37531
--webserver_port=38269
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39423
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:21.568460 31690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:21.568768 31690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:21.568900 31690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:21.576531 31690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:21.576658 31690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:21.576790 31690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:59:21.586913 31690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39423
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:36665
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=38269
--tserver_master_addrs=127.4.40.62:37531
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:21.588299 31690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:21.589690 31690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:21.603621 31701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:21.603654 31702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:21.603536 31709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:21.604493 31690 server_base.cc:1034] running on GCE node
I20260811 06:59:21.605047 31690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:21.605912 31690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:21.607137 31690 hybrid_clock.cc:648] HybridClock initialized: now 1786431561607087 us; error 62 us; skew 500 ppm
I20260811 06:59:21.607450 31690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:21 dist-test-slave-7vjv krb5kdc[29134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431561, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:21.611059 31690 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:59:21.612387 31690 webserver.cc:460] Webserver started at http://127.4.40.3:38269/ using document root <none> and password file <none>
I20260811 06:59:21.613013 31690 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:21.613113 31690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:21.618222 31690 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 06:59:21.620954 31718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:21.622213 31690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:59:21.622345 31690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "23763a99124c40fa982e4ffa1c5fe29f"
format_stamp: "Formatted at 2026-08-11 06:59:15 on dist-test-slave-7vjv"
server_key: "3ec70d95a1a04908ede8305dea97b2ab"
server_key_iv: "650213d819f8e6e275c2b871480772cd"
server_key_version: "encryptionkey@0"
I20260811 06:59:21.623239 31690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:21.640481 31690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:21.644341 31690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:21.644567 31690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:21.645432 31690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:21.646677 31690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:21.646798 31690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:21.646935 31690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:21.647018 31690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:21.681757 31690 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:36665
I20260811 06:59:21.681854 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:36665 every 8 connection(s)
I20260811 06:59:21.683245 31690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:59:21.686527 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 31690
I20260811 06:59:21.688213 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.686105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:44907)
0811 06:59:21.687044 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.687316 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":320,"threads_started":1}
W20260811 06:59:21.688912 31921 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:37531 (0 consecutive failures): Network error: Failed to ping master at 127.4.40.62:37531: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
I20260811 06:59:21.689697 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.689404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:40507)
0811 06:59:21.689534 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.689598 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 06:59:21.690505 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.690211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:56185)
0811 06:59:21.690324 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.690393 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 06:59:21.690752 31921 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 06:59:21.743391 31930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:21.742580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:45581)
0811 06:59:21.743082 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:21.743180 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":231,"threads_started":1}
I20260811 06:59:22.214838 31933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:22.214030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:37443)
0811 06:59:22.214503 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:22.214605 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":227,"threads_started":1}
I20260811 06:59:22.692203 31934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:22.691308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:56751)
0811 06:59:22.691782 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:22.691883 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":244,"threads_started":1}
I20260811 06:59:22.744946 31935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:22.744267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:38797)
0811 06:59:22.744690 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:22.744786 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":212,"threads_started":1}
I20260811 06:59:23.216471 31936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:23.215701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:42055)
0811 06:59:23.216210 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:23.216292 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":248,"threads_started":1}
I20260811 06:59:23.693746 31938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:23.693029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:37589)
0811 06:59:23.693489 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:23.693587 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":226,"threads_started":1}
I20260811 06:59:23.746734 31939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:23.745867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:39755)
0811 06:59:23.746417 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:23.746547 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":286,"threads_started":1}
I20260811 06:59:24.218092 31940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:24.217332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:45465)
0811 06:59:24.217794 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:24.217870 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":259,"threads_started":1}
I20260811 06:59:24.695533 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:24.694828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:59815)
0811 06:59:24.695283 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:24.695360 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":255,"threads_started":1}
I20260811 06:59:24.748411 31947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:24.747659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:45041)
0811 06:59:24.748165 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:24.748246 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":223,"threads_started":1}
I20260811 06:59:25.219817 31948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:25.218930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:40009)
0811 06:59:25.219526 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:25.219636 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":345,"threads_started":1}
I20260811 06:59:25.700862 31951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:25.696411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:42341)
0811 06:59:25.700552 (+ 4141us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:25.700640 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4031,"thread_start_us":3889,"threads_started":1}
I20260811 06:59:25.750047 31952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:25.749290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:39171)
0811 06:59:25.749783 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:25.749861 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":211,"threads_started":1}
I20260811 06:59:26.221494 31957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:26.220743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:47043)
0811 06:59:26.221228 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:26.221325 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":258,"threads_started":1}
I20260811 06:59:26.702514 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:26.701784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:50449)
0811 06:59:26.702268 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:26.702344 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":254,"threads_started":1}
I20260811 06:59:26.751612 31960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:26.750901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:35529)
0811 06:59:26.751340 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:26.751420 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260811 06:59:27.223623 31961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:27.222786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:40193)
0811 06:59:27.223362 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:27.223455 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":337,"threads_started":1}
I20260811 06:59:27.704411 31962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:27.703575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:54031)
0811 06:59:27.704151 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:27.704247 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":293,"threads_started":1}
I20260811 06:59:27.753371 31964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:27.752594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:33341)
0811 06:59:27.753091 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:27.753168 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":236,"threads_started":1}
I20260811 06:59:28.225407 31965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:28.224589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:49599)
0811 06:59:28.225103 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:28.225194 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":221,"threads_started":1}
I20260811 06:59:28.706125 31966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:28.705323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:42393)
0811 06:59:28.705824 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:28.705904 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":234,"threads_started":1}
I20260811 06:59:28.754878 31967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:28.754192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:48047)
0811 06:59:28.754635 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:28.754716 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":240,"threads_started":1}
I20260811 06:59:29.227269 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:29.226435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:60457)
0811 06:59:29.227025 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:29.227102 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":355,"threads_started":1}
I20260811 06:59:29.707803 31976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:29.706998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:56531)
0811 06:59:29.707496 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:29.707572 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":239,"threads_started":1}
I20260811 06:59:29.756633 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:29.755793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:51295)
0811 06:59:29.756346 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:29.756433 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":256,"threads_started":1}
I20260811 06:59:30.229048 31985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:30.228205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:56145)
0811 06:59:30.228751 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:30.228844 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":265,"threads_started":1}
I20260811 06:59:30.710331 31986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:30.709503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:51019)
0811 06:59:30.710016 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:30.710093 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":255,"threads_started":1}
I20260811 06:59:30.758247 31987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:30.757505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:41999)
0811 06:59:30.757954 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:30.758056 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":229,"threads_started":1}
I20260811 06:59:31.230719 31988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:31.229961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:40687)
0811 06:59:31.230469 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:31.230552 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":279,"threads_started":1}
I20260811 06:59:31.712177 31989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:31.711406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:59579)
0811 06:59:31.711883 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:31.711962 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":256,"threads_started":1}
I20260811 06:59:31.759899 31990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:31.759149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:39937)
0811 06:59:31.759625 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:31.759719 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":247,"threads_started":1}
I20260811 06:59:32.232451 31993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:32.231587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:49769)
0811 06:59:32.232075 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:32.232254 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":266,"threads_started":1}
I20260811 06:59:32.713891 31994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:32.713069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:44015)
0811 06:59:32.713588 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:32.713708 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":233,"threads_started":1}
I20260811 06:59:32.761454 31997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:32.760763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:45235)
0811 06:59:32.761222 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:32.761298 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":243,"threads_started":1}
I20260811 06:59:33.234030 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:33.233321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:47479)
0811 06:59:33.233791 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:33.233869 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":254,"threads_started":1}
I20260811 06:59:33.715583 32001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:33.714802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:40693)
0811 06:59:33.715332 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:33.715411 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":313,"threads_started":1}
I20260811 06:59:33.763175 32002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:33.762401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:56569)
0811 06:59:33.762882 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:33.762972 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":257,"threads_started":1}
I20260811 06:59:34.236223 32003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:34.235258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:49351)
0811 06:59:34.235872 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:34.235965 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":353,"threads_started":1}
I20260811 06:59:34.717515 32004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:34.716658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:59739)
0811 06:59:34.717236 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:34.717321 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":260,"threads_started":1}
I20260811 06:59:34.764791 32005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:34.763972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:55269)
0811 06:59:34.764527 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:34.764602 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":311,"threads_started":1}
I20260811 06:59:35.237833 32006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:35.237136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:50929)
0811 06:59:35.237562 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:35.237642 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":219,"threads_started":1}
I20260811 06:59:35.719509 32008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:35.718586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:49901)
0811 06:59:35.719139 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:35.719273 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":273,"threads_started":1}
I20260811 06:59:35.766553 32009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:35.765813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.1:46525)
0811 06:59:35.766274 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:35.766353 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":225,"threads_started":1}
I20260811 06:59:36.239861 32010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:36.239034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.2:37051)
0811 06:59:36.239550 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:36.239655 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":305,"threads_started":1}
I20260811 06:59:36.694900 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 31169
I20260811 06:59:36.712944 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:36.695889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35918 (local address 127.4.40.1:33191)
0811 06:59:36.696685 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:36.696703 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:36.696748 (+ 45us) server_negotiation.cc:407] Connection header received
0811 06:59:36.699197 (+ 2449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:36.699208 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:36.699660 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:36.699941 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:36.701415 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:36.702607 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:36.703766 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:36.704420 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:36.704555 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:36.707075 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:36.707082 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:36.707095 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:36.707132 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:36.709896 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:36.710565 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:36.710571 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:36.710577 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:36.710755 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:36.711099 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:36.711102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:36.711104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:36.711611 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:36.711824 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:36.712201 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 06:59:36.712652 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":440,"threads_started":1}
I20260811 06:59:36.714268 31447 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:36.721587 32014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:36.720700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:37531 (local address 127.4.40.3:43449)
0811 06:59:36.721221 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:36.721338 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:37531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":245,"threads_started":1}
I20260811 06:59:36.800932 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 31169
I20260811 06:59:36.835907 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 31463
I20260811 06:59:36.855279 32017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:36.837114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.4.40.2:38107)
0811 06:59:36.837683 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:36.837697 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:36.837746 (+ 49us) server_negotiation.cc:407] Connection header received
0811 06:59:36.839610 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:36.839621 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:36.840194 (+ 573us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:36.840589 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:36.842205 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:36.843517 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:36.844955 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:36.845631 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:36.845873 (+ 242us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:36.848798 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:36.848806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:36.848820 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:36.848870 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:36.851763 (+ 2893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:36.852644 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:36.852652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:36.852667 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:36.852841 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:36.853267 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:36.853271 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:36.853273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:36.853757 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:36.854096 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:36.854563 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 06:59:36.854974 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":264,"threads_started":1}
I20260811 06:59:36.856693 31676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:36.941622 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 31463
I20260811 06:59:36.971585 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 31690
I20260811 06:59:36.989744 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:36.972648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53590 (local address 127.4.40.3:36665)
0811 06:59:36.973286 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:36.973301 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:36.973356 (+ 55us) server_negotiation.cc:407] Connection header received
0811 06:59:36.974890 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:36.974899 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:36.975339 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:36.975698 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:36.977244 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:36.978504 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:36.979798 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:36.980527 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:36.980741 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:36.983386 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:36.983394 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:36.983408 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:36.983472 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:36.986297 (+ 2825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:36.986975 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:36.986982 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:36.986989 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:36.987158 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:36.987546 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:36.987553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:36.987556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:36.988087 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:36.988468 (+ 381us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:36.988962 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 06:59:36.989389 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":288,"threads_started":1}
I20260811 06:59:36.991230 31914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:37.074208 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 31690
2026-08-11T06:59:37Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25362 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 06:59:37 dist-test-slave-7vjv krb5kdc[32025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 06:59:37 dist-test-slave-7vjv krb5kdc[32025](info): set up 2 sockets
Aug 11 06:59:37 dist-test-slave-7vjv krb5kdc[32025](info): commencing operation
krb5kdc: starting...
W20260811 06:59:38.463435 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 06:59:38 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T06:59:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T06:59:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 06:59:38.765985 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42515
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:39.108700 32049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:39.109025 32049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:39.109193 32049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:39.116307 32049 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 06:59:39.116427 32049 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 06:59:39.116551 32049 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 06:59:39.116655 32049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 06:59:39.116751 32049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:39.116873 32049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:39.116959 32049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 06:59:39.117038 32049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 06:59:39.126777 32049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:39.128105 32049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:39.129467 32049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:39.141237 32060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:39.141399 32066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:39.141423 32061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:39.142024 32049 server_base.cc:1034] running on GCE node
I20260811 06:59:39.142863 32049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:39.144320 32049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:39.145567 32049 hybrid_clock.cc:648] HybridClock initialized: now 1786431579145498 us; error 38 us; skew 500 ppm
I20260811 06:59:39.145906 32049 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:39 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431579, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:39.154201 32049 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 06:59:39.155634 32049 webserver.cc:460] Webserver started at http://127.4.40.62:39835/ using document root <none> and password file <none>
I20260811 06:59:39.156411 32049 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:39.156564 32049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:39.156879 32049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:39.159946 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "3c4b0dd082124b59bc2e215de659709c"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "d9d85981ba739d0e4ed12200709f4f64"
server_key_iv: "50d6927c189499a27dfe00dbac13c6ad"
server_key_version: "encryptionkey@0"
I20260811 06:59:39.160833 32049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "3c4b0dd082124b59bc2e215de659709c"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "d9d85981ba739d0e4ed12200709f4f64"
server_key_iv: "50d6927c189499a27dfe00dbac13c6ad"
server_key_version: "encryptionkey@0"
I20260811 06:59:39.165844 32049 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 06:59:39.169173 32081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:39.170390 32049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 06:59:39.170576 32049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "3c4b0dd082124b59bc2e215de659709c"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "d9d85981ba739d0e4ed12200709f4f64"
server_key_iv: "50d6927c189499a27dfe00dbac13c6ad"
server_key_version: "encryptionkey@0"
I20260811 06:59:39.170792 32049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:39.201251 32049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:39.205158 32049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:39.205433 32049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:39.223838 32049 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:42515
I20260811 06:59:39.223873 32165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:42515 every 8 connection(s)
I20260811 06:59:39.225196 32049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 06:59:39.228794 32170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:39.234804 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 32049
I20260811 06:59:39.235087 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 06:59:39.235947 4256 external_mini_cluster.cc:1428] Setting key f3f273ab9059b72464fb082a5ab5654e
I20260811 06:59:39.240243 32170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap starting.
I20260811 06:59:39.243986 32170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:39.245203 32170 log.cc:824] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:39.248317 32170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: No bootstrap required, opened a new log
Aug 11 06:59:39 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:39.258839 32170 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 06:59:39.259347 32170 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:39.259466 32170 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4b0dd082124b59bc2e215de659709c, State: Initialized, Role: FOLLOWER
I20260811 06:59:39.260157 32170 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 06:59:39.260479 32170 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 06:59:39.260629 32170 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 06:59:39.260821 32170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:39.261077 32173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:39.238116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.4.40.62:42515)
0811 06:59:39.238944 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:39.238955 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:39.238996 (+ 41us) server_negotiation.cc:407] Connection header received
0811 06:59:39.240778 (+ 1782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:39.240800 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:39.241139 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:39.241432 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:39.243437 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:39.244890 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:39.246541 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:39.247179 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:39.247301 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:39.252146 (+ 4845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:39.252160 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:39.252177 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:39.252232 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:39.256440 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:39.257103 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:39.257111 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:39.257120 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:39.257362 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:39.257758 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:39.257764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:39.257767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:39.258452 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:39.258738 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:39.259067 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 06:59:39.259599 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":413,"threads_started":1}
I20260811 06:59:39.263262 32170 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 06:59:39.263751 32170 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4b0dd082124b59bc2e215de659709c; no voters:
I20260811 06:59:39.264393 32170 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 06:59:39.264532 32176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:39.266086 32176 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 LEADER]: Becoming Leader. State: Replica: 3c4b0dd082124b59bc2e215de659709c, State: Running, Role: LEADER
I20260811 06:59:39.266640 32170 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 06:59:39.266605 32176 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 06:59:39.270835 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3c4b0dd082124b59bc2e215de659709c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } } }
I20260811 06:59:39.270792 32179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4b0dd082124b59bc2e215de659709c. Latest consensus state: current_term: 1 leader_uuid: "3c4b0dd082124b59bc2e215de659709c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } } }
I20260811 06:59:39.271377 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:39.271436 32179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: This master's current role is: LEADER
I20260811 06:59:39.273069 32194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 06:59:39.279490 32194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 06:59:39.287791 32194 catalog_manager.cc:1347] Generated new cluster ID: 64bf19731d8f46738bb92706e643ed59
I20260811 06:59:39.287889 32194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 06:59:39.310463 32194 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 06:59:39.311771 32194 catalog_manager.cc:1504] Loading token signing keys...
I20260811 06:59:39.322491 32194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Generated new TSK 0
I20260811 06:59:39.323308 32194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 06:59:39.417769 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:42515
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:39.777369 32209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:39.777697 32209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:39.777801 32209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:39.784947 32209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:39.785064 32209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:39.785172 32209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 06:59:39.794726 32209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:39.795908 32209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:39.797260 32209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:39.810761 32217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:39.810734 32219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:39.810942 32222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:39.811808 32209 server_base.cc:1034] running on GCE node
I20260811 06:59:39.812500 32209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:39.813341 32209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:39.814556 32209 hybrid_clock.cc:648] HybridClock initialized: now 1786431579814520 us; error 28 us; skew 500 ppm
I20260811 06:59:39.814882 32209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:39 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431579, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:39.818164 32209 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 06:59:39.819751 32209 webserver.cc:460] Webserver started at http://127.4.40.1:40823/ using document root <none> and password file <none>
I20260811 06:59:39.820500 32209 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:39.820638 32209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:39.820982 32209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:39.824061 32209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "3a3c12b81dbb40dc87114c3830531f93"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "8291a95fe233efd77d61ea916c7a8158"
server_key_iv: "0fe89224da58d1bba4e9a50583380856"
server_key_version: "encryptionkey@0"
I20260811 06:59:39.824990 32209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "3a3c12b81dbb40dc87114c3830531f93"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "8291a95fe233efd77d61ea916c7a8158"
server_key_iv: "0fe89224da58d1bba4e9a50583380856"
server_key_version: "encryptionkey@0"
I20260811 06:59:39.829802 32209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 06:59:39.833259 32231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:39.834537 32209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 06:59:39.834721 32209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "3a3c12b81dbb40dc87114c3830531f93"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "8291a95fe233efd77d61ea916c7a8158"
server_key_iv: "0fe89224da58d1bba4e9a50583380856"
server_key_version: "encryptionkey@0"
I20260811 06:59:39.834918 32209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:39.846364 32209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:39.850271 32209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:39.850554 32209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:39.851478 32209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:39.852797 32209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:39.852905 32209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:39.853050 32209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:39.853111 32209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:39.888033 32209 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:37389
I20260811 06:59:39.888170 32447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:37389 every 8 connection(s)
I20260811 06:59:39.889441 32209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 06:59:39.899323 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 32209
I20260811 06:59:39.899569 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 06:59:39.900323 4256 external_mini_cluster.cc:1428] Setting key a8bb8375c819c5fd574bc0bb4650ab72
Aug 11 06:59:39 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431579, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:39.910904 32450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:39.892131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:49099 (local address 127.4.40.62:42515)
0811 06:59:39.892573 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:39.892577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:39.893533 (+ 956us) server_negotiation.cc:407] Connection header received
0811 06:59:39.895376 (+ 1843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:39.895382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:39.895478 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:39.895603 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:39.897176 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:39.897916 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:39.899447 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:39.899868 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:39.900036 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:39.903851 (+ 3815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:39.903856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:39.903859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:39.903888 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:39.907218 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:39.908201 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:39.908208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:39.908213 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:39.908344 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:39.908863 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:39.908870 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:39.908875 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:39.909200 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:39.909333 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:39.910220 (+ 887us) server_negotiation.cc:299] Negotiation successful
0811 06:59:39.910478 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":225,"threads_started":1}
I20260811 06:59:39.912168 32451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:39.892667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:49099)
0811 06:59:39.893358 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:39.893401 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:39.895109 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:39.895843 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:39.895851 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:39.896430 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:39.897002 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:39.897020 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:39.898084 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:39.898089 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:39.899300 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:39.899313 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:39.900152 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:39.900162 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:39.900405 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:39.901197 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:39.901228 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:39.903640 (+ 2412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:39.907372 (+ 3732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:39.907382 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:39.907399 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:39.908053 (+ 654us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:39.908454 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:39.908461 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:39.908467 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:39.908728 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:39.909332 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:39.909342 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:39.909782 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 06:59:39.910094 (+ 312us) client_negotiation.cc:240] Negotiation successful
0811 06:59:39.910701 (+ 607us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":269,"threads_started":1}
I20260811 06:59:39.913669 32448 heartbeater.cc:347] Connected to a master server at 127.4.40.62:42515
I20260811 06:59:39.913995 32448 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:39.914834 32448 heartbeater.cc:511] Master 127.4.40.62:42515 requested a full tablet report, sending...
I20260811 06:59:39.917325 32103 ts_manager.cc:194] Registered new tserver with Master: 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389)
I20260811 06:59:39.918684 32103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:49099
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 06:59:39.971257 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:42515
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:40.315764 32458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:40.316156 32458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:40.316362 32458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:40.323905 32458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:40.324128 32458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:40.324339 32458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 06:59:40.334726 32458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:40.336639 32458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:40.338265 32458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:40.352880 32476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:40.352847 32471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:40.352926 32470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:40.353690 32458 server_base.cc:1034] running on GCE node
I20260811 06:59:40.354413 32458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:40.355332 32458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:40.356575 32458 hybrid_clock.cc:648] HybridClock initialized: now 1786431580356530 us; error 63 us; skew 500 ppm
I20260811 06:59:40.356887 32458 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:40 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:40.360030 32458 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 06:59:40.361418 32458 webserver.cc:460] Webserver started at http://127.4.40.2:43469/ using document root <none> and password file <none>
I20260811 06:59:40.362169 32458 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:40.362323 32458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:40.362648 32458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:40.365578 32458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "abaaa4db50f24a20be98945dd6778dd8"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "30a00aa6705eb3d36faa51ec609bdd5f"
server_key_iv: "cad61068c73226190d9356f5f2488af1"
server_key_version: "encryptionkey@0"
I20260811 06:59:40.366357 32458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "abaaa4db50f24a20be98945dd6778dd8"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "30a00aa6705eb3d36faa51ec609bdd5f"
server_key_iv: "cad61068c73226190d9356f5f2488af1"
server_key_version: "encryptionkey@0"
I20260811 06:59:40.371381 32458 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 06:59:40.374877 32483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:40.376370 32458 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:59:40.376545 32458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "abaaa4db50f24a20be98945dd6778dd8"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "30a00aa6705eb3d36faa51ec609bdd5f"
server_key_iv: "cad61068c73226190d9356f5f2488af1"
server_key_version: "encryptionkey@0"
I20260811 06:59:40.376742 32458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:40.402387 32458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:40.407038 32458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:40.407348 32458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:40.408356 32458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:40.409698 32458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:40.409771 32458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:40.409945 32458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:40.409999 32458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:40.442175 32458 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:39733
I20260811 06:59:40.442221 32686 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:39733 every 8 connection(s)
I20260811 06:59:40.443576 32458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 06:59:40.452724 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 32458
I20260811 06:59:40.452940 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 06:59:40.453444 4256 external_mini_cluster.cc:1428] Setting key 1a8a208c5a7499f945807bc64ab1f775
Aug 11 06:59:40 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:40.464696 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:40.446187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:54587 (local address 127.4.40.62:42515)
0811 06:59:40.446664 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:40.446668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:40.447491 (+ 823us) server_negotiation.cc:407] Connection header received
0811 06:59:40.449227 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:40.449231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:40.449298 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:40.449392 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:40.450848 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:40.451558 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:40.453347 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:40.453942 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:40.454090 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:40.457691 (+ 3601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:40.457700 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:40.457709 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:40.457765 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:40.461543 (+ 3778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:40.462234 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:40.462242 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:40.462247 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:40.462418 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:40.462809 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:40.462816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:40.462821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:40.463191 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:40.463347 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:40.463983 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 06:59:40.464244 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":233,"threads_started":1}
I20260811 06:59:40.465296 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:40.446642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:54587)
0811 06:59:40.447333 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:40.447371 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:40.448979 (+ 1608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:40.449584 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:40.449592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:40.450074 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:40.450613 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:40.450629 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:40.451697 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:40.451700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:40.453208 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:40.453225 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:40.454208 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:40.454218 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:40.454468 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:40.455199 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:40.455230 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:40.457469 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:40.461675 (+ 4206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:40.461681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:40.461692 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:40.462108 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:40.462524 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:40.462527 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:40.462529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:40.462697 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:40.463312 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:40.463317 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:40.463602 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 06:59:40.463820 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 06:59:40.464220 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":278,"threads_started":1}
I20260811 06:59:40.466821 32689 heartbeater.cc:347] Connected to a master server at 127.4.40.62:42515
I20260811 06:59:40.467175 32689 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:40.468046 32689 heartbeater.cc:511] Master 127.4.40.62:42515 requested a full tablet report, sending...
I20260811 06:59:40.469784 32103 ts_manager.cc:194] Registered new tserver with Master: abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733)
I20260811 06:59:40.471053 32103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:54587
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 06:59:40.523772 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:42515
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 06:59:40.867414 32695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 06:59:40.867724 32695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 06:59:40.867846 32695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 06:59:40.874854 32695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 06:59:40.875000 32695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 06:59:40.875125 32695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 06:59:40.885497 32695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 06:59:40.887194 32695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 06:59:40.888692 32695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 06:59:40.901933 32705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:40.902038 32706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 06:59:40.902034 32712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 06:59:40.902777 32695 server_base.cc:1034] running on GCE node
I20260811 06:59:40.903340 32695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 06:59:40.904361 32695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 06:59:40.905583 32695 hybrid_clock.cc:648] HybridClock initialized: now 1786431580905502 us; error 71 us; skew 500 ppm
I20260811 06:59:40.905890 32695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 06:59:40 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 06:59:40.909155 32695 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 06:59:40.910458 32695 webserver.cc:460] Webserver started at http://127.4.40.3:40639/ using document root <none> and password file <none>
I20260811 06:59:40.911088 32695 fs_manager.cc:359] Metadata directory not provided
I20260811 06:59:40.911192 32695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 06:59:40.911453 32695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 06:59:40.914355 32695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "a501381321d48676446ebf32b5c24388"
server_key_iv: "3643cc3a2d1a5bb377467cf7d2440cfc"
server_key_version: "encryptionkey@0"
I20260811 06:59:40.915082 32695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "a501381321d48676446ebf32b5c24388"
server_key_iv: "3643cc3a2d1a5bb377467cf7d2440cfc"
server_key_version: "encryptionkey@0"
I20260811 06:59:40.920187 32695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 06:59:40.922345 32448 heartbeater.cc:503] Master 127.4.40.62:42515 was elected leader, sending a full tablet report...
I20260811 06:59:40.923794 32726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:40.925222 32695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 06:59:40.925402 32695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "a501381321d48676446ebf32b5c24388"
server_key_iv: "3643cc3a2d1a5bb377467cf7d2440cfc"
server_key_version: "encryptionkey@0"
I20260811 06:59:40.925628 32695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 06:59:40.958685 32695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 06:59:40.962779 32695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 06:59:40.963074 32695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 06:59:40.964051 32695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 06:59:40.965569 32695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 06:59:40.965687 32695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:40.965839 32695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 06:59:40.965910 32695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 06:59:40.999759 32695 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:45139
I20260811 06:59:40.999809 461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:45139 every 8 connection(s)
I20260811 06:59:41.001134 32695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 06:59:41.005852 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 32695
I20260811 06:59:41.006052 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 06:59:41.006496 4256 external_mini_cluster.cc:1428] Setting key 8f2b12390bfeac5c6e4495189fe869a2
Aug 11 06:59:41 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431580, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 06:59:41.021281 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.003689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:59037 (local address 127.4.40.62:42515)
0811 06:59:41.004201 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.004208 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.005184 (+ 976us) server_negotiation.cc:407] Connection header received
0811 06:59:41.007235 (+ 2051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.007242 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.007347 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.007482 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:41.009494 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.010336 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.011828 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.012269 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.012359 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.015570 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:41.015574 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:41.015577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:41.015606 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:41.018355 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.018986 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.018989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.018991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.019099 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.019445 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.019448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.019450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.019728 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:41.019848 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.020635 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.020858 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":221,"threads_started":1}
I20260811 06:59:41.022243 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.004330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:59037)
0811 06:59:41.005024 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:41.005063 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:41.006898 (+ 1835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:41.007758 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:41.007770 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:41.008539 (+ 769us) client_negotiation.cc:189] Negotiated authn=SASL
0811 06:59:41.009321 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.009346 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.010480 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.010484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.011697 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.011708 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.012454 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.012459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.012643 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.013255 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 06:59:41.013280 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 06:59:41.015384 (+ 2104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 06:59:41.018467 (+ 3083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:41.018473 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:41.018483 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:41.018874 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:41.019190 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 06:59:41.019193 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 06:59:41.019196 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 06:59:41.019356 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 06:59:41.019835 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 06:59:41.019840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 06:59:41.020215 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 06:59:41.020503 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 06:59:41.021044 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":268,"threads_started":1}
I20260811 06:59:41.023583 464 heartbeater.cc:347] Connected to a master server at 127.4.40.62:42515
I20260811 06:59:41.023830 464 heartbeater.cc:464] Registering TS with master...
I20260811 06:59:41.024564 464 heartbeater.cc:511] Master 127.4.40.62:42515 requested a full tablet report, sending...
I20260811 06:59:41.025947 32103 ts_manager.cc:194] Registered new tserver with Master: e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
I20260811 06:59:41.026708 32103 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:59037
I20260811 06:59:41.034336 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 06:59:41.051532 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.039105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.4.40.62:42515)
0811 06:59:41.039309 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.039317 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.039733 (+ 416us) server_negotiation.cc:407] Connection header received
0811 06:59:41.039863 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.039867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.039939 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.040026 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:41.041533 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.042227 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.043722 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.044186 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.044300 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.046760 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:41.046765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:41.046768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:41.046796 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:41.049276 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.049922 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.049925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.049927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.050041 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.050367 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.050370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.050372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.050673 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:41.050857 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.051127 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.051302 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 06:59:41.055326 32103 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 06:59:41.062021 32103 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 06:59:41.096285 491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.073504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:54446)
0811 06:59:41.085064 (+ 11560us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:41.085110 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:41.085290 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:41.086940 (+ 1650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:41.086951 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:41.087000 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:41.087519 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.087536 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.090604 (+ 3068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.090608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.093034 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.093055 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.095066 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.095073 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.095295 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.095359 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 06:59:41.095498 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 06:59:41.095881 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11265,"thread_start_us":15215,"threads_started":1}
I20260811 06:59:41.097239 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.073769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54446 (local address 127.4.40.1:37389)
0811 06:59:41.086112 (+ 12343us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.086124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.086162 (+ 38us) server_negotiation.cc:407] Connection header received
0811 06:59:41.086301 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.086313 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.086601 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.086876 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:41.087784 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.090416 (+ 2632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.093273 (+ 2857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.094800 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.094992 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.095253 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.096792 (+ 1539us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.096969 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12085,"thread_start_us":11867,"threads_started":1}
I20260811 06:59:41.097958 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.074371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.4.40.2:39733)
0811 06:59:41.088322 (+ 13951us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.088328 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.088621 (+ 293us) server_negotiation.cc:407] Connection header received
0811 06:59:41.088777 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.088782 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.089040 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.089240 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:41.090872 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.092311 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.094897 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.096586 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.096733 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.096910 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.097441 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.097624 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13803,"thread_start_us":13640,"threads_started":1}
I20260811 06:59:41.098976 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.089429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.4.40.3:45139)
0811 06:59:41.090090 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.090098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.091326 (+ 1228us) server_negotiation.cc:407] Connection header received
0811 06:59:41.091465 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.091472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.091662 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.091854 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:41.092674 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.093664 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.095826 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.097703 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.097805 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.098033 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.098618 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.098745 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":320,"threads_started":1}
I20260811 06:59:41.099093 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.089100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:45139 (local address 127.0.0.1:51074)
0811 06:59:41.091186 (+ 2086us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:41.091205 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:41.091335 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:41.091861 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:41.091865 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:41.091890 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:41.092501 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.092511 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.093808 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.093816 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.095639 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.095654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.098244 (+ 2590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.098253 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.098438 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.098468 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 06:59:41.098571 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 06:59:41.098692 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1962,"mutex_wait_us":25,"thread_start_us":1305,"threads_started":1}
I20260811 06:59:41.100703 487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.073923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:39733 (local address 127.0.0.1:57030)
0811 06:59:41.088512 (+ 14589us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:41.088530 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:41.088651 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:41.089398 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:41.089405 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:41.089431 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:41.090680 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.090697 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.092483 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.092488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.094723 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.094738 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.097049 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.097055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.097229 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.097266 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 06:59:41.100292 (+ 3026us) client_negotiation.cc:240] Negotiation successful
0811 06:59:41.100394 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14476,"mutex_wait_us":1,"thread_start_us":13215,"threads_started":1}
I20260811 06:59:41.101176 32330 tablet_service.cc:1467] Processing CreateTablet for tablet 4a68ea52f7844b119e2ad498b3dd4979 (DEFAULT_TABLE table=test-table [id=b7529c3763214ae492a08e015eaf1563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:41.102780 32330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a68ea52f7844b119e2ad498b3dd4979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:41.102703 349 tablet_service.cc:1467] Processing CreateTablet for tablet 4a68ea52f7844b119e2ad498b3dd4979 (DEFAULT_TABLE table=test-table [id=b7529c3763214ae492a08e015eaf1563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:41.104162 349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a68ea52f7844b119e2ad498b3dd4979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:41.104984 32571 tablet_service.cc:1467] Processing CreateTablet for tablet 4a68ea52f7844b119e2ad498b3dd4979 (DEFAULT_TABLE table=test-table [id=b7529c3763214ae492a08e015eaf1563]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 06:59:41.106025 32571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a68ea52f7844b119e2ad498b3dd4979. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 06:59:41.116326 496 tablet_bootstrap.cc:492] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Bootstrap starting.
I20260811 06:59:41.118257 498 tablet_bootstrap.cc:492] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Bootstrap starting.
I20260811 06:59:41.120220 496 tablet_bootstrap.cc:654] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:41.121575 496 log.cc:824] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:41.121857 498 tablet_bootstrap.cc:654] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:41.123098 498 log.cc:824] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:41.124114 499 tablet_bootstrap.cc:492] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Bootstrap starting.
I20260811 06:59:41.125677 496 tablet_bootstrap.cc:492] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: No bootstrap required, opened a new log
I20260811 06:59:41.125954 496 ts_tablet_manager.cc:1397] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 06:59:41.127419 498 tablet_bootstrap.cc:492] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: No bootstrap required, opened a new log
I20260811 06:59:41.127629 499 tablet_bootstrap.cc:654] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Neither blocks nor log segments found. Creating new log.
I20260811 06:59:41.127724 498 ts_tablet_manager.cc:1397] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260811 06:59:41.128952 499 log.cc:824] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Log is configured to *not* fsync() on all Append() calls
I20260811 06:59:41.132265 499 tablet_bootstrap.cc:492] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: No bootstrap required, opened a new log
I20260811 06:59:41.132524 499 ts_tablet_manager.cc:1397] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260811 06:59:41.134083 496 raft_consensus.cc:348] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.134434 496 raft_consensus.cc:374] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:41.134569 496 raft_consensus.cc:729] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abaaa4db50f24a20be98945dd6778dd8, State: Initialized, Role: FOLLOWER
I20260811 06:59:41.135229 496 consensus_queue.cc:260] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.138267 498 raft_consensus.cc:348] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.138676 498 raft_consensus.cc:374] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:41.138777 498 raft_consensus.cc:729] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3c12b81dbb40dc87114c3830531f93, State: Initialized, Role: FOLLOWER
I20260811 06:59:41.139406 498 consensus_queue.cc:260] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.141235 498 ts_tablet_manager.cc:1428] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
W20260811 06:59:41.141835 32449 tablet.cc:2369] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:41.143381 496 ts_tablet_manager.cc:1428] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Time spent starting tablet: real 0.017s user 0.014s sys 0.004s
I20260811 06:59:41.143807 32689 heartbeater.cc:503] Master 127.4.40.62:42515 was elected leader, sending a full tablet report...
I20260811 06:59:41.144479 499 raft_consensus.cc:348] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.144920 499 raft_consensus.cc:374] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 06:59:41.145054 499 raft_consensus.cc:729] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56cff34d6f04b7b8b2e69ce3745cfbe, State: Initialized, Role: FOLLOWER
I20260811 06:59:41.145691 499 consensus_queue.cc:260] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.146904 464 heartbeater.cc:503] Master 127.4.40.62:42515 was elected leader, sending a full tablet report...
I20260811 06:59:41.147543 499 ts_tablet_manager.cc:1428] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Time spent starting tablet: real 0.015s user 0.005s sys 0.008s
W20260811 06:59:41.196053 32690 tablet.cc:2369] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:41.211146 510 raft_consensus.cc:482] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 06:59:41.211406 510 raft_consensus.cc:504] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.213065 510 leader_election.cc:288] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389), abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733)
I20260811 06:59:41.219364 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.213494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:55909 (local address 127.4.40.2:39733)
0811 06:59:41.213713 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.213720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.213743 (+ 23us) server_negotiation.cc:407] Connection header received
0811 06:59:41.213989 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.213996 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.214091 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.214195 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:41.214640 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.215604 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.217466 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.218556 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.218659 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.218729 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.219087 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.219178 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:59:41.219481 470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.213330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:39733 (local address 127.4.40.3:55909)
0811 06:59:41.213627 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:41.213652 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:41.213790 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:41.214221 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:41.214224 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:41.214250 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:41.214491 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.214503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.215732 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.215738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.217306 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.217319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.218711 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.218717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.218909 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.218947 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 06:59:41.219059 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 06:59:41.219195 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 06:59:41.220332 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.213593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38631 (local address 127.4.40.1:37389)
0811 06:59:41.213783 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.213789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.214388 (+ 599us) server_negotiation.cc:407] Connection header received
0811 06:59:41.214557 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.214565 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.214653 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.214717 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 06:59:41.215326 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.216256 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.218265 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.219338 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.219489 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.219627 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.219950 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.220085 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 06:59:41.220340 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.213418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.4.40.3:38631)
0811 06:59:41.214252 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0811 06:59:41.214268 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 06:59:41.214409 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 06:59:41.214778 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 06:59:41.214783 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 06:59:41.214807 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 06:59:41.215165 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.215177 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.216416 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.216420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.218078 (+ 1658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 06:59:41.218094 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.219592 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.219598 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 06:59:41.219767 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.219798 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 06:59:41.219890 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 06:59:41.219976 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":544,"threads_started":1}
I20260811 06:59:41.220747 32601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a68ea52f7844b119e2ad498b3dd4979" candidate_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abaaa4db50f24a20be98945dd6778dd8" is_pre_election: true
I20260811 06:59:41.221256 32601 raft_consensus.cc:2454] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56cff34d6f04b7b8b2e69ce3745cfbe in term 0.
I20260811 06:59:41.221315 32365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a68ea52f7844b119e2ad498b3dd4979" candidate_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3c12b81dbb40dc87114c3830531f93" is_pre_election: true
I20260811 06:59:41.221894 32365 raft_consensus.cc:2454] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56cff34d6f04b7b8b2e69ce3745cfbe in term 0.
I20260811 06:59:41.221971 32728 leader_election.cc:302] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe; no voters:
I20260811 06:59:41.222380 510 raft_consensus.cc:2781] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 06:59:41.222546 510 raft_consensus.cc:482] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 06:59:41.222692 510 raft_consensus.cc:3037] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:41.225262 510 raft_consensus.cc:504] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.226158 510 leader_election.cc:288] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [CANDIDATE]: Term 1 election: Requested vote from peers 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389), abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733)
I20260811 06:59:41.226442 32601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a68ea52f7844b119e2ad498b3dd4979" candidate_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "abaaa4db50f24a20be98945dd6778dd8"
I20260811 06:59:41.226649 32601 raft_consensus.cc:3037] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:41.228914 32601 raft_consensus.cc:2454] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56cff34d6f04b7b8b2e69ce3745cfbe in term 1.
I20260811 06:59:41.229347 32728 leader_election.cc:302] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe; no voters:
I20260811 06:59:41.229635 510 raft_consensus.cc:2781] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 06:59:41.230105 510 raft_consensus.cc:686] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 LEADER]: Becoming Leader. State: Replica: e56cff34d6f04b7b8b2e69ce3745cfbe, State: Running, Role: LEADER
I20260811 06:59:41.230728 510 consensus_queue.cc:237] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } }
I20260811 06:59:41.234567 32365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a68ea52f7844b119e2ad498b3dd4979" candidate_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3c12b81dbb40dc87114c3830531f93"
I20260811 06:59:41.234814 32365 raft_consensus.cc:3037] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 06:59:41.235587 32103 catalog_manager.cc:5495] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe reported cstate change: term changed from 0 to 1, leader changed from <none> to e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3). New cstate: current_term: 1 leader_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } health_report { overall_health: UNKNOWN } } }
I20260811 06:59:41.237877 32365 raft_consensus.cc:2454] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56cff34d6f04b7b8b2e69ce3745cfbe in term 1.
W20260811 06:59:41.252795 468 tablet.cc:2369] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 06:59:41.274302 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.269069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51076 (local address 127.4.40.3:45139)
0811 06:59:41.269246 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.269250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.269369 (+ 119us) server_negotiation.cc:407] Connection header received
0811 06:59:41.269538 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.269541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.269616 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.269738 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 06:59:41.270076 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.270768 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.272281 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.272692 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.272782 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.273245 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 06:59:41.273757 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 06:59:41.273932 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.273976 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.274063 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 06:59:41.282845 32365 raft_consensus.cc:1261] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Refusing update from remote peer e56cff34d6f04b7b8b2e69ce3745cfbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:41.282845 32601 raft_consensus.cc:1261] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Refusing update from remote peer e56cff34d6f04b7b8b2e69ce3745cfbe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 06:59:41.283965 517 consensus_queue.cc:1035] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:41.284425 510 consensus_queue.cc:1035] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 06:59:41.300688 528 mvcc.cc:205] Tried to move back new op lower bound from 7317223756923654144 to 7317223756724953088. Current Snapshot: MvccSnapshot[applied={T|T < 7317223756923654144}]
I20260811 06:59:41.310590 32102 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:41.310916 32102 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38682:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 06:59:41.314651 32102 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Sending DeleteTablet for 3 replicas of tablet 4a68ea52f7844b119e2ad498b3dd4979
I20260811 06:59:41.315868 32571 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a68ea52f7844b119e2ad498b3dd4979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:41 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:57030
I20260811 06:59:41.316048 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 32209
I20260811 06:59:41.316072 349 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a68ea52f7844b119e2ad498b3dd4979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:41 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:51074
I20260811 06:59:41.316066 32330 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a68ea52f7844b119e2ad498b3dd4979 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 06:59:41 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:54446
I20260811 06:59:41.327313 557 tablet_replica.cc:330] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: stopping tablet replica
I20260811 06:59:41.327952 557 raft_consensus.cc:2229] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:41.328734 557 raft_consensus.cc:2258] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:41.329607 555 tablet_replica.cc:330] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: stopping tablet replica
I20260811 06:59:41.331151 555 raft_consensus.cc:2229] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 06:59:41.331753 555 raft_consensus.cc:2258] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 06:59:41.332656 556 tablet_replica.cc:330] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: stopping tablet replica
I20260811 06:59:41.333338 557 ts_tablet_manager.cc:1905] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:41.334451 555 ts_tablet_manager.cc:1905] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:41.336607 556 raft_consensus.cc:2229] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 LEADER]: Raft consensus shutting down.
I20260811 06:59:41.337445 556 raft_consensus.cc:2258] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 06:59:41 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 06:59:41.341715 557 ts_tablet_manager.cc:1918] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:41.342000 557 log.cc:1196] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/4a68ea52f7844b119e2ad498b3dd4979
I20260811 06:59:41.342686 557 ts_tablet_manager.cc:1939] T 4a68ea52f7844b119e2ad498b3dd4979 P 3a3c12b81dbb40dc87114c3830531f93: Deleting consensus metadata
I20260811 06:59:41.344879 32086 catalog_manager.cc:4841] TS 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389): tablet 4a68ea52f7844b119e2ad498b3dd4979 (table test-table [id=b7529c3763214ae492a08e015eaf1563]) successfully deleted
I20260811 06:59:41.345034 555 ts_tablet_manager.cc:1918] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:41.345184 555 log.cc:1196] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/4a68ea52f7844b119e2ad498b3dd4979
I20260811 06:59:41.345799 556 ts_tablet_manager.cc:1905] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 06:59:41.349217 489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.317605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54456 (local address 127.4.40.1:37389)
0811 06:59:41.317826 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.317836 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.326941 (+ 9105us) server_negotiation.cc:407] Connection header received
0811 06:59:41.327336 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.327344 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.327428 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.327516 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:41.330123 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.331282 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.334511 (+ 3229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.335331 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.335433 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.341613 (+ 6180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:41.341629 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:41.341646 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:41.341702 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:41.345829 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.346762 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.346771 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.346775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.346937 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.347391 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.347396 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.347399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.347871 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:41.348193 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.348589 (+ 396us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.348828 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 06:59:41.349896 32440 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:41.352859 555 ts_tablet_manager.cc:1939] T 4a68ea52f7844b119e2ad498b3dd4979 P abaaa4db50f24a20be98945dd6778dd8: Deleting consensus metadata
I20260811 06:59:41.354766 32082 catalog_manager.cc:4841] TS abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): tablet 4a68ea52f7844b119e2ad498b3dd4979 (table test-table [id=b7529c3763214ae492a08e015eaf1563]) successfully deleted
I20260811 06:59:41.356199 556 ts_tablet_manager.cc:1918] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 06:59:41.356375 556 log.cc:1196] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/4a68ea52f7844b119e2ad498b3dd4979
I20260811 06:59:41.356951 556 ts_tablet_manager.cc:1939] T 4a68ea52f7844b119e2ad498b3dd4979 P e56cff34d6f04b7b8b2e69ce3745cfbe: Deleting consensus metadata
I20260811 06:59:41.358948 32086 catalog_manager.cc:4841] TS e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): tablet 4a68ea52f7844b119e2ad498b3dd4979 (table test-table [id=b7529c3763214ae492a08e015eaf1563]) successfully deleted
I20260811 06:59:41.441764 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 32209
I20260811 06:59:41.477561 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 32458
Aug 11 06:59:41 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 06:59:41.496368 492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.478473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.4.40.2:39733)
0811 06:59:41.478673 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.478679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.478703 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:59:41.478898 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.478904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.479011 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.479117 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:41.480981 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.482218 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.484297 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.484970 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.485092 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.490325 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:41.490336 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:41.490350 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:41.490379 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:41.493444 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.494229 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.494236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.494239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.494375 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.494769 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.494773 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.494775 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.495142 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:41.495459 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.495781 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.495998 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 06:59:41.497244 32677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:41.587455 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 32458
I20260811 06:59:41.617213 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 32695
Aug 11 06:59:41 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 06:59:41.635564 495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 06:59:41.618176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51092 (local address 127.4.40.3:45139)
0811 06:59:41.618387 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 06:59:41.618394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 06:59:41.618418 (+ 24us) server_negotiation.cc:407] Connection header received
0811 06:59:41.618492 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 06:59:41.618499 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 06:59:41.618595 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 06:59:41.618725 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 06:59:41.620073 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.621476 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.622793 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 06:59:41.623474 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 06:59:41.623606 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 06:59:41.628152 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 06:59:41.628169 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 06:59:41.628191 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 06:59:41.628245 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 06:59:41.632590 (+ 4345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.633235 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.633245 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.633248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.633449 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 06:59:41.633862 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 06:59:41.633867 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 06:59:41.633870 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 06:59:41.634382 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 06:59:41.634741 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0811 06:59:41.634905 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 06:59:41.635134 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 06:59:41.636611 456 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:41.760408 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 32695
I20260811 06:59:41.791991 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 32049
I20260811 06:59:41.793224 32157 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 06:59:41.905099 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 32049
I20260811 07:00:01.935328 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42515
--webserver_interface=127.4.40.62
--webserver_port=39835
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:02.276124 568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:02.276401 568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:02.276678 568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:02.283820 568 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:00:02.283902 568 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:00:02.283965 568 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:00:02.284034 568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:00:02.284126 568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:02.284265 568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:02.284324 568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:00:02.284415 568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:00:02.293756 568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=39835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:02.295009 568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:02.296327 568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:02.322165 582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:02.322196 578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:02.322113 577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:02.323252 568 server_base.cc:1034] running on GCE node
I20260811 07:00:02.324275 568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:02.325852 568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:02.327131 568 hybrid_clock.cc:648] HybridClock initialized: now 1786431602327032 us; error 68 us; skew 500 ppm
I20260811 07:00:02.327684 568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:02 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:02.331552 568 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:00:02.333525 568 webserver.cc:460] Webserver started at http://127.4.40.62:39835/ using document root <none> and password file <none>
I20260811 07:00:02.334400 568 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:02.334513 568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:02.341344 568 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 07:00:02.345624 594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:02.347474 568 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 07:00:02.347673 568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "3c4b0dd082124b59bc2e215de659709c"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "d9d85981ba739d0e4ed12200709f4f64"
server_key_iv: "50d6927c189499a27dfe00dbac13c6ad"
server_key_version: "encryptionkey@0"
I20260811 07:00:02.349140 568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:02.372077 568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:02.376688 568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:02.376938 568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:02.395879 568 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:42515
I20260811 07:00:02.395910 682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:42515 every 8 connection(s)
I20260811 07:00:02.397446 568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:00:02.399408 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 568
I20260811 07:00:02.400346 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:37389
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:42515
--webserver_port=40823
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:00:02.404685 686 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:00:02.407987 686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap starting.
I20260811 07:00:02.431576 686 log.cc:824] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:02.444222 686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:00:02.444782 686 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap complete.
I20260811 07:00:02.453857 686 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:02.454788 686 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4b0dd082124b59bc2e215de659709c, State: Initialized, Role: FOLLOWER
I20260811 07:00:02.455461 686 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:02.455729 686 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:00:02.455852 686 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:00:02.456054 686 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 1 FOLLOWER]: Advancing to term 2
I20260811 07:00:02.459733 686 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:02.460341 686 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4b0dd082124b59bc2e215de659709c; no voters:
I20260811 07:00:02.461018 686 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 07:00:02.461156 694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Leader election won for term 2
I20260811 07:00:02.462385 694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 LEADER]: Becoming Leader. State: Replica: 3c4b0dd082124b59bc2e215de659709c, State: Running, Role: LEADER
I20260811 07:00:02.462919 694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:02.463320 686 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:00:02.465579 700 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4b0dd082124b59bc2e215de659709c. Latest consensus state: current_term: 2 leader_uuid: "3c4b0dd082124b59bc2e215de659709c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } } }
I20260811 07:00:02.465986 700 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:02.466615 697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3c4b0dd082124b59bc2e215de659709c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } } }
I20260811 07:00:02.466969 697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:02.476900 710 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:00:02.486624 710 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:00:02.488270 710 catalog_manager.cc:1259] Loaded cluster ID: 64bf19731d8f46738bb92706e643ed59
I20260811 07:00:02.488452 710 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:00:02.492972 710 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:00:02.494614 710 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Loaded TSK: 0
I20260811 07:00:02.512552 736 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:00:02.535534 710 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Generated new TSK 1
I20260811 07:00:02.535995 710 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 07:00:02.788800 692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:02.789157 692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:02.789335 692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:02.796571 692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:02.796727 692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:02.796963 692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:00:02.806571 692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:37389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=40823
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:02.807971 692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:02.809435 692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:02.827638 749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:02.827752 746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:02.828148 751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:02.828795 692 server_base.cc:1034] running on GCE node
I20260811 07:00:02.829420 692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:02.830447 692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:02.831681 692 hybrid_clock.cc:648] HybridClock initialized: now 1786431602831627 us; error 46 us; skew 500 ppm
I20260811 07:00:02.832015 692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:02 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:02.835335 692 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 07:00:02.837170 692 webserver.cc:460] Webserver started at http://127.4.40.1:40823/ using document root <none> and password file <none>
I20260811 07:00:02.837864 692 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:02.838022 692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:02.843403 692 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:00:02.846349 768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:02.847751 692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:00:02.847916 692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "3a3c12b81dbb40dc87114c3830531f93"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "8291a95fe233efd77d61ea916c7a8158"
server_key_iv: "0fe89224da58d1bba4e9a50583380856"
server_key_version: "encryptionkey@0"
I20260811 07:00:02.848913 692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:02.881338 692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:02.885438 692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:02.885740 692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:02.886783 692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:02.888280 692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:02.888420 692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:02.888567 692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:02.888662 692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:02.921697 692 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:37389
I20260811 07:00:02.921787 970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:37389 every 8 connection(s)
I20260811 07:00:02.923193 692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:00:02.929616 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 692
I20260811 07:00:02.930275 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:39733
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:42515
--webserver_port=43469
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:00:02 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431602, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:02.954543 975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:02.926233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:40539)
0811 07:00:02.927047 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:02.927088 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:02.928914 (+ 1826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:02.930292 (+ 1378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:02.930304 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:02.930914 (+ 610us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:02.931415 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:02.931431 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:02.933165 (+ 1734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:02.933169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:02.934450 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:02.934472 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:02.935477 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:02.935486 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:02.935834 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:02.943437 (+ 7603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:02.943472 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:02.946479 (+ 3007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:02.949878 (+ 3399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:02.949887 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:02.949900 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:02.950327 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:02.950802 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:02.950806 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:02.950810 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:02.950995 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:02.951813 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:02.951820 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:02.952192 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 07:00:02.952623 (+ 431us) client_negotiation.cc:240] Negotiation successful
0811 07:00:02.953211 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":337,"threads_started":1}
I20260811 07:00:02.954928 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:02.926016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:40539 (local address 127.4.40.62:42515)
0811 07:00:02.926623 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:02.926642 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:02.927217 (+ 575us) server_negotiation.cc:407] Connection header received
0811 07:00:02.929425 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:02.929437 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:02.929781 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:02.930148 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:02.931632 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:02.932960 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:02.934604 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:02.935229 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:02.935422 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:02.946789 (+ 11367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:02.946796 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:02.946811 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:02.946847 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:02.949721 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:02.950475 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:02.950482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:02.950487 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:02.950670 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:02.951126 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:02.951132 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:02.951135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:02.951666 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:02.951867 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:02.952958 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0811 07:00:02.953637 (+ 679us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":243,"threads_started":1}
I20260811 07:00:02.957587 971 heartbeater.cc:347] Connected to a master server at 127.4.40.62:42515
I20260811 07:00:02.957902 971 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:02.958613 971 heartbeater.cc:511] Master 127.4.40.62:42515 requested a full tablet report, sending...
I20260811 07:00:02.960824 619 ts_manager.cc:194] Registered new tserver with Master: 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389)
I20260811 07:00:02.963106 619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:40539
W20260811 07:00:03.275728 976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:03.276046 976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:03.276232 976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:03.283993 976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:03.284164 976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:03.284336 976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:00:03.293880 976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:39733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=43469
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:03.295166 976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:03.296553 976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:03.309624 986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:03.309664 988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:03.309700 990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:03.310459 976 server_base.cc:1034] running on GCE node
I20260811 07:00:03.311065 976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:03.311986 976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:03.313212 976 hybrid_clock.cc:648] HybridClock initialized: now 1786431603313186 us; error 67 us; skew 500 ppm
I20260811 07:00:03.313529 976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:03 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431603, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:03.317164 976 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 07:00:03.318714 976 webserver.cc:460] Webserver started at http://127.4.40.2:43469/ using document root <none> and password file <none>
I20260811 07:00:03.319437 976 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:03.319625 976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:03.325105 976 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:00:03.327900 1007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:03.329329 976 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:00:03.329514 976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "abaaa4db50f24a20be98945dd6778dd8"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "30a00aa6705eb3d36faa51ec609bdd5f"
server_key_iv: "cad61068c73226190d9356f5f2488af1"
server_key_version: "encryptionkey@0"
I20260811 07:00:03.330508 976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:03.359372 976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:03.363842 976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:03.364185 976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:03.365150 976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:03.366549 976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:03.366670 976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:03.366803 976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:03.366889 976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:03.402192 976 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:39733
I20260811 07:00:03.402275 1218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:39733 every 8 connection(s)
I20260811 07:00:03.403594 976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:00:03.405267 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 976
I20260811 07:00:03.405942 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:45139
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:42515
--webserver_port=40639
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 07:00:03 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431603, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:03.424856 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.408284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:36595 (local address 127.4.40.62:42515)
0811 07:00:03.408453 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.408459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.408482 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:00:03.409919 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.409925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.410067 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.410185 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:03.411596 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.412395 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.413956 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.414399 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.414540 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.418284 (+ 3744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:03.418290 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:03.418295 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:03.418335 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:03.421719 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.422476 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.422481 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.422483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.422608 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.423073 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.423079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.423082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.423358 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:03.423529 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:03.424254 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 07:00:03.424501 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 07:00:03.425856 1222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.406796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:36595)
0811 07:00:03.407590 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:03.407644 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:03.409644 (+ 2000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:03.410428 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:03.410435 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:03.410912 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:03.411431 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.411446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.412558 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.412564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.413801 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.413812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.414629 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.414637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.414872 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.415553 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:03.415589 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:03.417992 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:03.421866 (+ 3874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:03.421876 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:03.421892 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:03.422327 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:03.422726 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:03.422731 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:03.422735 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:03.422937 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:03.423507 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:03.423514 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:03.423847 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 07:00:03.424158 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 07:00:03.424696 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":320,"threads_started":1}
I20260811 07:00:03.427294 1220 heartbeater.cc:347] Connected to a master server at 127.4.40.62:42515
I20260811 07:00:03.427621 1220 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:03.428628 1220 heartbeater.cc:511] Master 127.4.40.62:42515 requested a full tablet report, sending...
I20260811 07:00:03.430274 619 ts_manager.cc:194] Registered new tserver with Master: abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733)
I20260811 07:00:03.431262 619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:36595
W20260811 07:00:03.783838 1226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:03.784209 1226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:03.784392 1226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:03.791600 1226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:03.791757 1226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:03.791940 1226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:00:03.801839 1226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:45139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=40639
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:03.803169 1226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:03.804713 1226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:03.818571 1237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:03.818650 1240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:03.818545 1236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:03.819444 1226 server_base.cc:1034] running on GCE node
I20260811 07:00:03.820061 1226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:03.821049 1226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:03.822263 1226 hybrid_clock.cc:648] HybridClock initialized: now 1786431603822223 us; error 30 us; skew 500 ppm
I20260811 07:00:03.822580 1226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:03 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431603, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:03.825639 1226 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 07:00:03.827162 1226 webserver.cc:460] Webserver started at http://127.4.40.3:40639/ using document root <none> and password file <none>
I20260811 07:00:03.827862 1226 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:03.828032 1226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:03.833220 1226 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:00:03.836045 1251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:03.837590 1226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:00:03.837764 1226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "a501381321d48676446ebf32b5c24388"
server_key_iv: "3643cc3a2d1a5bb377467cf7d2440cfc"
server_key_version: "encryptionkey@0"
I20260811 07:00:03.838716 1226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:03.854460 1226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:03.858194 1226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:03.858491 1226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:03.859449 1226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:03.860816 1226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:03.860945 1226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:03.861078 1226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:03.861171 1226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:03.895351 1226 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:45139
I20260811 07:00:03.895442 1488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:45139 every 8 connection(s)
I20260811 07:00:03.896940 1226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:00:03.904556 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 1226
Aug 11 07:00:03 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431603, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:03.917196 974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.899555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:45793 (local address 127.4.40.62:42515)
0811 07:00:03.899755 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.899760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.900895 (+ 1135us) server_negotiation.cc:407] Connection header received
0811 07:00:03.902723 (+ 1828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.902728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.902816 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.902928 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:03.904491 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.905432 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.907322 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.907747 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.907879 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.911345 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:03.911350 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:03.911354 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:03.911384 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:03.914146 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.914846 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.914851 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.914853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.914985 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.915425 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.915428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.915430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.915717 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:03.915872 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:03.916614 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 07:00:03.916824 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 07:00:03.918129 1491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.899965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:45793)
0811 07:00:03.900731 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:03.900776 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:03.902442 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:03.903169 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:03.903176 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:03.903625 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:03.904325 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.904344 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.905642 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.905647 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.907176 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.907188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.907998 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.908006 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.908261 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.908914 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:03.908935 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:03.911145 (+ 2210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:03.914290 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:03.914298 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:03.914311 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:03.914712 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:03.915102 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:03.915109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:03.915113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:03.915299 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:03.915844 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:03.915852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:03.916227 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 07:00:03.916465 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 07:00:03.917000 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":347,"threads_started":1}
I20260811 07:00:03.919696 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.905623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57528 (local address 127.4.40.62:42515)
0811 07:00:03.906456 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.906461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.906482 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:00:03.906517 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.906520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.906565 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.906675 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:03.908223 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.909120 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.910456 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.910921 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.911060 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.913881 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:03.913886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:03.913888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:03.913928 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:03.916634 (+ 2706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.917562 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.917566 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.917568 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.917685 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.918092 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.918098 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.918101 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.918431 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:03.918710 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:03.919070 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 07:00:03.919280 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":577,"threads_started":1}
I20260811 07:00:03.919826 1489 heartbeater.cc:347] Connected to a master server at 127.4.40.62:42515
I20260811 07:00:03.920190 1489 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:03.921048 1489 heartbeater.cc:511] Master 127.4.40.62:42515 requested a full tablet report, sending...
I20260811 07:00:03.922639 619 ts_manager.cc:194] Registered new tserver with Master: e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
I20260811 07:00:03.923712 619 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:45793
I20260811 07:00:03.925302 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:00:03.932129 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.926327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57536 (local address 127.4.40.62:42515)
0811 07:00:03.926473 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.926476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.926963 (+ 487us) server_negotiation.cc:407] Connection header received
0811 07:00:03.927217 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.927221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.927333 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.927490 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:00:03.927954 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.928684 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.930171 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.930588 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.930721 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.931217 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:00:03.931478 (+ 261us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 07:00:03.931830 (+ 352us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57536: token expired
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 07:00:03.932271 1494 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57536: token expired
W20260811 07:00:03.932735 471 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.40.62:42515)
I20260811 07:00:03.946019 1494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.933186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57542 (local address 127.4.40.62:42515)
0811 07:00:03.933324 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.933328 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.933358 (+ 30us) server_negotiation.cc:407] Connection header received
0811 07:00:03.933482 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.933485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.933544 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.933618 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:03.935120 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.935861 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.937388 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.937842 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.937936 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.940456 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:03.940461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:03.940464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:03.940493 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:03.942923 (+ 2430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.943533 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.943537 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.943539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.943691 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:03.944017 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:03.944021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:03.944022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:03.944379 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:03.944651 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:03.945484 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 07:00:03.945713 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:00:03.950479 619 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:00:03.959609 619 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:00:03.966452 971 heartbeater.cc:503] Master 127.4.40.62:42515 was elected leader, sending a full tablet report...
I20260811 07:00:03.988677 1511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.977862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:45139 (local address 127.0.0.1:49792)
0811 07:00:03.978599 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:03.978616 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:03.978725 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:03.979874 (+ 1149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:03.979882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:03.979930 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:03.980489 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.980511 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.982786 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.982791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.985367 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.985382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.987540 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.987547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.987750 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.987808 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 07:00:03.987962 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 07:00:03.988191 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":27,"spinlock_wait_cycles":6400,"thread_start_us":362,"threads_started":1}
I20260811 07:00:03.988734 1512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.978246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.4.40.3:45139)
0811 07:00:03.979146 (+ 900us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.979159 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.979202 (+ 43us) server_negotiation.cc:407] Connection header received
0811 07:00:03.979322 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.979333 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.979640 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.979873 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:03.980681 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.982652 (+ 1971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.985548 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.987309 (+ 1761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.987499 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.987814 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:03.988069 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 07:00:03.988325 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":871,"threads_started":1}
I20260811 07:00:03.993566 1345 tablet_service.cc:1467] Processing CreateTablet for tablet 7b011940b294421eb4cf194c098484d3 (DEFAULT_TABLE table=test-table [id=f8701d87f93348ed9a17808961406c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:03.995445 1345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b011940b294421eb4cf194c098484d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:03.989291 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.976494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34676)
0811 07:00:03.977706 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:03.977727 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:03.977888 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:03.979874 (+ 1986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:03.979882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:03.979923 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:03.980745 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.980757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.983094 (+ 2337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.983099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.985658 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.985676 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.988298 (+ 2622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.988307 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:03.988508 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:03.988540 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 07:00:03.988634 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 07:00:03.988774 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"mutex_wait_us":286,"spinlock_wait_cycles":896,"thread_start_us":954,"threads_started":1}
I20260811 07:00:04.001314 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.976793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.4.40.1:37389)
0811 07:00:03.977443 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.977453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.979337 (+ 1884us) server_negotiation.cc:407] Connection header received
0811 07:00:03.979445 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.979453 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.979679 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.980369 (+ 690us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:03.980927 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.982933 (+ 2006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.985874 (+ 2941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.988069 (+ 2195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.000453 (+ 12384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.000707 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.000786 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.000979 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":334,"threads_started":1}
I20260811 07:00:04.004482 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.974313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:39733 (local address 127.0.0.1:50146)
0811 07:00:03.975053 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:03.975088 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:03.975244 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:03.994947 (+ 19703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:03.994953 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:03.994994 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:03.995332 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:03.995344 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.997528 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:03.997534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:04.001746 (+ 4212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:04.001759 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.003666 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.003675 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:04.003865 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.003907 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 07:00:04.003995 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 07:00:04.004155 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":362,"threads_started":1}
I20260811 07:00:04.005589 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:03.975595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50146 (local address 127.4.40.2:39733)
0811 07:00:03.994253 (+ 18658us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:03.994262 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:03.994290 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:00:03.994385 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:03.994393 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:03.994591 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:03.995504 (+ 913us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:03.995741 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:03.997363 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.001906 (+ 4543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.003424 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.004827 (+ 1403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.005029 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.005095 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.005251 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18488,"thread_start_us":19120,"threads_started":1}
I20260811 07:00:04.005805 855 tablet_service.cc:1467] Processing CreateTablet for tablet 7b011940b294421eb4cf194c098484d3 (DEFAULT_TABLE table=test-table [id=f8701d87f93348ed9a17808961406c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:04.007215 855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b011940b294421eb4cf194c098484d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:04.009285 1096 tablet_service.cc:1467] Processing CreateTablet for tablet 7b011940b294421eb4cf194c098484d3 (DEFAULT_TABLE table=test-table [id=f8701d87f93348ed9a17808961406c4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:04.010649 1096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7b011940b294421eb4cf194c098484d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:04.026094 1516 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Bootstrap starting.
I20260811 07:00:04.027022 1515 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Bootstrap starting.
I20260811 07:00:04.030112 1516 tablet_bootstrap.cc:654] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:04.030531 1515 tablet_bootstrap.cc:654] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:04.031232 1517 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Bootstrap starting.
I20260811 07:00:04.031517 1516 log.cc:824] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:04.031819 1515 log.cc:824] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:04.036304 1517 tablet_bootstrap.cc:654] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:04.038054 1517 log.cc:824] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:04.041711 1515 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: No bootstrap required, opened a new log
I20260811 07:00:04.042044 1515 ts_tablet_manager.cc:1397] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260811 07:00:04.042984 1516 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: No bootstrap required, opened a new log
I20260811 07:00:04.043251 1516 ts_tablet_manager.cc:1397] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.003s
I20260811 07:00:04.044943 1517 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: No bootstrap required, opened a new log
I20260811 07:00:04.045210 1517 ts_tablet_manager.cc:1397] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.003s
I20260811 07:00:04.049793 1515 raft_consensus.cc:348] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.050163 1515 raft_consensus.cc:374] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:04.050307 1515 raft_consensus.cc:729] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e56cff34d6f04b7b8b2e69ce3745cfbe, State: Initialized, Role: FOLLOWER
I20260811 07:00:04.050910 1515 consensus_queue.cc:260] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.052002 1489 heartbeater.cc:503] Master 127.4.40.62:42515 was elected leader, sending a full tablet report...
I20260811 07:00:04.052748 1515 ts_tablet_manager.cc:1428] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 07:00:04.053833 1516 raft_consensus.cc:348] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.054265 1516 raft_consensus.cc:374] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:04.054365 1516 raft_consensus.cc:729] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3c12b81dbb40dc87114c3830531f93, State: Initialized, Role: FOLLOWER
I20260811 07:00:04.054916 1516 consensus_queue.cc:260] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.056676 1516 ts_tablet_manager.cc:1428] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Time spent starting tablet: real 0.013s user 0.014s sys 0.001s
I20260811 07:00:04.057396 1517 raft_consensus.cc:348] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.057823 1517 raft_consensus.cc:374] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:04.057935 1517 raft_consensus.cc:729] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abaaa4db50f24a20be98945dd6778dd8, State: Initialized, Role: FOLLOWER
I20260811 07:00:04.058537 1517 consensus_queue.cc:260] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.059903 1220 heartbeater.cc:503] Master 127.4.40.62:42515 was elected leader, sending a full tablet report...
I20260811 07:00:04.060338 1517 ts_tablet_manager.cc:1428] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
W20260811 07:00:04.148691 1490 tablet.cc:2369] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:00:04.159631 1221 tablet.cc:2369] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:00:04.176082 972 tablet.cc:2369] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:04.233207 1529 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:04.233543 1529 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.235424 1529 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
I20260811 07:00:04.245491 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.236018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:47215 (local address 127.4.40.1:37389)
0811 07:00:04.236252 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:04.236257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:04.238532 (+ 2275us) server_negotiation.cc:407] Connection header received
0811 07:00:04.238846 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:04.238854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:04.238951 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:04.239069 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:04.239775 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.240794 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.243591 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.244474 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.244577 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.244650 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.245087 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.245217 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 07:00:04.245558 1512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.238714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:36999 (local address 127.4.40.3:45139)
0811 07:00:04.238963 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:04.238972 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:04.239403 (+ 431us) server_negotiation.cc:407] Connection header received
0811 07:00:04.239544 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:04.239550 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:04.239642 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:04.239729 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:04.240386 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.241451 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.243794 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.244643 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.244770 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.244872 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.245187 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.245324 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 07:00:04.245607 1531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.235767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.4.40.2:47215)
0811 07:00:04.238390 (+ 2623us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.238415 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:04.238668 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:04.239099 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:04.239110 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:04.239152 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:04.239590 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:04.239604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.240927 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.240932 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:04.243434 (+ 2502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:04.243446 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.244705 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.244714 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:04.244905 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.244940 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 07:00:04.245031 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 07:00:04.245144 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2454,"mutex_wait_us":68,"thread_start_us":2234,"threads_started":1}
I20260811 07:00:04.245671 1532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.238601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:45139 (local address 127.4.40.2:36999)
0811 07:00:04.239285 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.239298 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:04.239405 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:04.239774 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:04.239779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:04.239806 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:04.240220 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:04.240235 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.241677 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.241682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:04.243643 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:04.243657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.244844 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.244850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:04.244987 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.245041 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 07:00:04.245136 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:00:04.245246 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":25,"thread_start_us":340,"threads_started":1}
I20260811 07:00:04.246634 1392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b011940b294421eb4cf194c098484d3" candidate_uuid: "abaaa4db50f24a20be98945dd6778dd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" is_pre_election: true
I20260811 07:00:04.246639 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b011940b294421eb4cf194c098484d3" candidate_uuid: "abaaa4db50f24a20be98945dd6778dd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3c12b81dbb40dc87114c3830531f93" is_pre_election: true
I20260811 07:00:04.247131 896 raft_consensus.cc:2454] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abaaa4db50f24a20be98945dd6778dd8 in term 0.
I20260811 07:00:04.247131 1392 raft_consensus.cc:2454] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abaaa4db50f24a20be98945dd6778dd8 in term 0.
I20260811 07:00:04.247787 1015 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93, abaaa4db50f24a20be98945dd6778dd8; no voters:
I20260811 07:00:04.248211 1529 raft_consensus.cc:2781] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:00:04.248332 1529 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:00:04.248443 1529 raft_consensus.cc:3037] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:04.250664 1529 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.251241 1529 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [CANDIDATE]: Term 1 election: Requested vote from peers 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
I20260811 07:00:04.251714 896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b011940b294421eb4cf194c098484d3" candidate_uuid: "abaaa4db50f24a20be98945dd6778dd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a3c12b81dbb40dc87114c3830531f93"
I20260811 07:00:04.251729 1392 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7b011940b294421eb4cf194c098484d3" candidate_uuid: "abaaa4db50f24a20be98945dd6778dd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe"
I20260811 07:00:04.251892 896 raft_consensus.cc:3037] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:04.251911 1392 raft_consensus.cc:3037] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:04.254343 896 raft_consensus.cc:2454] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abaaa4db50f24a20be98945dd6778dd8 in term 1.
I20260811 07:00:04.254518 1392 raft_consensus.cc:2454] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abaaa4db50f24a20be98945dd6778dd8 in term 1.
I20260811 07:00:04.254827 1015 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93, abaaa4db50f24a20be98945dd6778dd8; no voters:
I20260811 07:00:04.255120 1529 raft_consensus.cc:2781] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:04.255569 1529 raft_consensus.cc:686] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 1 LEADER]: Becoming Leader. State: Replica: abaaa4db50f24a20be98945dd6778dd8, State: Running, Role: LEADER
I20260811 07:00:04.256034 1529 consensus_queue.cc:237] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:04.260780 619 catalog_manager.cc:5495] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2). New cstate: current_term: 1 leader_uuid: "abaaa4db50f24a20be98945dd6778dd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } health_report { overall_health: UNKNOWN } } }
I20260811 07:00:04.293438 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.288391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.4.40.2:39733)
0811 07:00:04.288589 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:04.288594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:04.288613 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:00:04.288774 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:04.288778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:04.288860 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:04.288990 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:00:04.289320 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.290021 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.291476 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.291881 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.291975 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.292470 (+ 495us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:00:04.292961 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:00:04.293085 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.293155 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.293222 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 07:00:04.301682 1392 raft_consensus.cc:1261] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Refusing update from remote peer abaaa4db50f24a20be98945dd6778dd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:04.301733 896 raft_consensus.cc:1261] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Refusing update from remote peer abaaa4db50f24a20be98945dd6778dd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:04.302613 1533 consensus_queue.cc:1035] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:04.303006 1529 consensus_queue.cc:1035] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:04.329861 617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:04.330222 617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:04.334221 617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Sending DeleteTablet for 3 replicas of tablet 7b011940b294421eb4cf194c098484d3
I20260811 07:00:04.335942 1345 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b011940b294421eb4cf194c098484d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:04 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:49792
I20260811 07:00:04.336779 1562 tablet_replica.cc:330] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: stopping tablet replica
I20260811 07:00:04.337186 1562 raft_consensus.cc:2229] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:04.337620 1562 raft_consensus.cc:2258] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:04.339321 1562 ts_tablet_manager.cc:1905] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:04.340023 855 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b011940b294421eb4cf194c098484d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:04 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:34676
I20260811 07:00:04.341450 1096 tablet_service.cc:1514] Processing DeleteTablet for tablet 7b011940b294421eb4cf194c098484d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:04 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:50146
I20260811 07:00:04.342854 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 692
I20260811 07:00:04.350502 1562 ts_tablet_manager.cc:1918] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:04.350708 1562 log.cc:1196] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/7b011940b294421eb4cf194c098484d3
I20260811 07:00:04.351176 1562 ts_tablet_manager.cc:1939] T 7b011940b294421eb4cf194c098484d3 P e56cff34d6f04b7b8b2e69ce3745cfbe: Deleting consensus metadata
I20260811 07:00:04.353097 599 catalog_manager.cc:4841] TS e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): tablet 7b011940b294421eb4cf194c098484d3 (table test-table [id=f8701d87f93348ed9a17808961406c4a]) successfully deleted
I20260811 07:00:04.362923 1506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.343672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34684 (local address 127.4.40.1:37389)
0811 07:00:04.343863 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:04.343870 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:04.343967 (+ 97us) server_negotiation.cc:407] Connection header received
0811 07:00:04.344213 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:04.344219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:04.344310 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:04.344421 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:04.346284 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.347034 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.348847 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.349286 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.349426 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.356549 (+ 7123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:04.356572 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:04.356585 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:04.356674 (+ 89us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:04.359749 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:04.360611 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:04.360626 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:04.360629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:04.360765 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:04.361172 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:04.361177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:04.361182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:04.361547 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:04.361878 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.362210 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.362434 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:00:04.363629 964 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:04.366966 1563 tablet_replica.cc:330] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: stopping tablet replica
I20260811 07:00:04.367739 1563 raft_consensus.cc:2229] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:00:04.368574 1563 raft_consensus.cc:2258] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:04.370898 1563 ts_tablet_manager.cc:1905] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:04.378481 1563 ts_tablet_manager.cc:1918] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:04.378657 1563 log.cc:1196] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/7b011940b294421eb4cf194c098484d3
I20260811 07:00:04.379315 1563 ts_tablet_manager.cc:1939] T 7b011940b294421eb4cf194c098484d3 P abaaa4db50f24a20be98945dd6778dd8: Deleting consensus metadata
I20260811 07:00:04.381307 595 catalog_manager.cc:4841] TS abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): tablet 7b011940b294421eb4cf194c098484d3 (table test-table [id=f8701d87f93348ed9a17808961406c4a]) successfully deleted
I20260811 07:00:04.470486 1565 tablet_replica.cc:330] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: stopping tablet replica
I20260811 07:00:04.470779 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 692
I20260811 07:00:04.497771 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 976
W20260811 07:00:04.497737 599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:37389 (error 108)
I20260811 07:00:04.499882 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.499395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34698)
0811 07:00:04.499568 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.499698 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 07:00:04.501219 599 catalog_manager.cc:4584] TS 3a3c12b81dbb40dc87114c3830531f93 (127.4.40.1:37389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7b011940b294421eb4cf194c098484d3: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
I20260811 07:00:04.514110 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.499020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50162 (local address 127.4.40.2:39733)
0811 07:00:04.499236 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:04.499244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:04.499265 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:00:04.499337 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:04.499357 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:04.499447 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:04.499560 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:04.501175 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.502410 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.503677 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.504383 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.504511 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.507318 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:04.507335 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:04.507361 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:04.507407 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:04.511274 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:04.511946 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:04.511955 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:04.511959 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:04.512163 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:04.512512 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:04.512517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:04.512521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:04.513003 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:04.513343 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.513515 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.513732 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 07:00:04.514868 1209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:04.554821 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.554335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34714)
0811 07:00:04.554577 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.554678 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 07:00:04.555910 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.555589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34718)
0811 07:00:04.555713 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.555780 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 07:00:04.616621 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.616049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34726)
0811 07:00:04.616358 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.616448 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260811 07:00:04.617689 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.617356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34732)
0811 07:00:04.617487 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.617582 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:00:04.628405 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 976
I20260811 07:00:04.659123 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 1226
I20260811 07:00:04.672452 1512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.659877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.4.40.3:45139)
0811 07:00:04.660047 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:04.660051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:04.660153 (+ 102us) server_negotiation.cc:407] Connection header received
0811 07:00:04.660345 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:04.660350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:04.660426 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:04.660520 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:04.661915 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.662620 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.663905 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:04.664351 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:04.664445 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:04.667092 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:04.667103 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:04.667118 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:04.667151 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:04.670005 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:04.670653 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:04.670660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:04.670662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:04.670811 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:04.671136 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:04.671139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:04.671140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:04.671483 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:04.671692 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:04.671963 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 07:00:04.672186 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:00:04.673115 1480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:04.698108 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.697566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34738)
0811 07:00:04.697790 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.697903 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 07:00:04.699180 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.698753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34742)
0811 07:00:04.698957 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.699043 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 07:00:04.763870 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 1226
I20260811 07:00:04.790755 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 568
I20260811 07:00:04.791852 677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:04.908010 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:04.907412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37389 (local address 127.0.0.1:34750)
0811 07:00:04.907636 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:04.907797 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 07:00:04.908021 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 568
I20260811 07:00:24.943758 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42515
--webserver_interface=127.4.40.62
--webserver_port=39835
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:42515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:25.302628 1578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:25.302958 1578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:25.303148 1578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:25.310330 1578 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 07:00:25.310470 1578 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 07:00:25.310585 1578 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 07:00:25.310698 1578 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:00:25.310804 1578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:25.310909 1578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:25.310999 1578 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:00:25.311082 1578 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:00:25.320806 1578 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:42515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=39835
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:25.322109 1578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:25.323475 1578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:25.336067 1591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:25.336107 1593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:25.336512 1578 server_base.cc:1034] running on GCE node
W20260811 07:00:25.336915 1588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:25.337811 1578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:25.339123 1578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:25.340344 1578 hybrid_clock.cc:648] HybridClock initialized: now 1786431625340289 us; error 24 us; skew 500 ppm
I20260811 07:00:25.340694 1578 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:25 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:25.343809 1578 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:00:25.345355 1578 webserver.cc:460] Webserver started at http://127.4.40.62:39835/ using document root <none> and password file <none>
I20260811 07:00:25.346088 1578 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:25.346206 1578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:25.351737 1578 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20260811 07:00:25.354635 1603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:25.355916 1578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 07:00:25.356087 1578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "3c4b0dd082124b59bc2e215de659709c"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "d9d85981ba739d0e4ed12200709f4f64"
server_key_iv: "50d6927c189499a27dfe00dbac13c6ad"
server_key_version: "encryptionkey@0"
I20260811 07:00:25.357146 1578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:25.377111 1578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:25.381479 1578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:25.381789 1578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:25.399320 1578 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:42515
I20260811 07:00:25.399371 1698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:42515 every 8 connection(s)
I20260811 07:00:25.400877 1578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:00:25.407894 1700 sys_catalog.cc:263] Verifying existing consensus state
I20260811 07:00:25.408887 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 1578
I20260811 07:00:25.409792 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:37389
--local_ip_for_outbound_sockets=127.4.40.1
--tserver_master_addrs=127.4.40.62:42515
--webserver_port=40823
--webserver_interface=127.4.40.1
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:00:25.411342 1700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap starting.
I20260811 07:00:25.432209 1700 log.cc:824] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:25.448598 1700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:00:25.449153 1700 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Bootstrap complete.
I20260811 07:00:25.458343 1700 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:25.459259 1700 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c4b0dd082124b59bc2e215de659709c, State: Initialized, Role: FOLLOWER
I20260811 07:00:25.459936 1700 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:25.460254 1700 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:00:25.460407 1700 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:00:25.460600 1700 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 2 FOLLOWER]: Advancing to term 3
I20260811 07:00:25.464560 1700 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:25.465030 1700 leader_election.cc:302] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3c4b0dd082124b59bc2e215de659709c; no voters:
I20260811 07:00:25.465746 1700 leader_election.cc:288] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 07:00:25.465883 1709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 3 FOLLOWER]: Leader election won for term 3
I20260811 07:00:25.467160 1709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [term 3 LEADER]: Becoming Leader. State: Replica: 3c4b0dd082124b59bc2e215de659709c, State: Running, Role: LEADER
I20260811 07:00:25.467705 1709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } }
I20260811 07:00:25.467971 1700 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:00:25.470546 1711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3c4b0dd082124b59bc2e215de659709c. Latest consensus state: current_term: 3 leader_uuid: "3c4b0dd082124b59bc2e215de659709c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } } }
I20260811 07:00:25.470578 1710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3c4b0dd082124b59bc2e215de659709c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3c4b0dd082124b59bc2e215de659709c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42515 } } }
I20260811 07:00:25.470978 1711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:25.471028 1710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:25.482038 1717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:00:25.491021 1717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:00:25.493359 1717 catalog_manager.cc:1259] Loaded cluster ID: 64bf19731d8f46738bb92706e643ed59
I20260811 07:00:25.493461 1717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:00:25.497458 1717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:00:25.499096 1717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Loaded TSK: 0
I20260811 07:00:25.499240 1717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3c4b0dd082124b59bc2e215de659709c: Loaded TSK: 1
F20260811 07:00:25.500381 1717 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0710ad39ad google::LogMessage::Fail() at ??:0
@ 0x7f0710ad688a google::LogMessage::SendToLog() at ??:0
@ 0x7f0710ad34de google::LogMessage::Flush() at ??:0
@ 0x7f0710ad4a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f07148377d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f071482673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f07265021c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0726501783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f072658d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f072658d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f07265d75e9 std::function<>::operator()() at ??:0
@ 0x7f0726505cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0726503c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f072658b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f072658b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f071371f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f071372291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0713722799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f07136fdeaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f07169d76db start_thread at ??:0
@ 0x7f070f96d61f clone at ??:0
W20260811 07:00:25.857676 1702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:25.857972 1702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:25.858085 1702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:25.865242 1702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:25.865337 1702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:25.865444 1702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:00:25.874975 1702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:37389
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=40823
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:25.876169 1702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:25.877434 1702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:25.890384 1787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:25.890461 1789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:25.890425 1794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:25.891535 1702 server_base.cc:1034] running on GCE node
I20260811 07:00:25.892267 1702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:25.893253 1702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:25.894459 1702 hybrid_clock.cc:648] HybridClock initialized: now 1786431625894405 us; error 42 us; skew 500 ppm
I20260811 07:00:25.894802 1702 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:25 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:25.898077 1702 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 07:00:25.899606 1702 webserver.cc:460] Webserver started at http://127.4.40.1:40823/ using document root <none> and password file <none>
I20260811 07:00:25.900379 1702 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:25.900532 1702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:25.905751 1702 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 07:00:25.908722 1808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:25.909991 1702 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:25.910158 1702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "3a3c12b81dbb40dc87114c3830531f93"
format_stamp: "Formatted at 2026-08-11 06:59:39 on dist-test-slave-7vjv"
server_key: "8291a95fe233efd77d61ea916c7a8158"
server_key_iv: "0fe89224da58d1bba4e9a50583380856"
server_key_version: "encryptionkey@0"
I20260811 07:00:25.911118 1702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:25.923889 1702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:25.928134 1702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:25.928408 1702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:25.929364 1702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:25.931114 1820 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 07:00:25.936725 1702 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 07:00:25.936883 1702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 07:00:25.937043 1702 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 07:00:25.940222 1702 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 07:00:25.940323 1702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:25.940631 1820 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Bootstrap starting.
I20260811 07:00:25.968641 1820 log.cc:824] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:25.983017 1820 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 07:00:25.983876 1820 tablet_bootstrap.cc:492] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Bootstrap complete.
I20260811 07:00:25.985138 1702 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:37389
I20260811 07:00:25.985239 2009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:37389 every 8 connection(s)
I20260811 07:00:25.985572 1820 ts_tablet_manager.cc:1397] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Time spent bootstrapping tablet: real 0.045s user 0.031s sys 0.012s
I20260811 07:00:25.986704 1702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:00:25.992715 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:25.990164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:56823)
0811 07:00:25.991176 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:25.991470 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":418,"threads_started":1}
W20260811 07:00:25.993579 2014 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:42515 (0 consecutive failures): Network error: Failed to ping master at 127.4.40.62:42515: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
I20260811 07:00:25.994490 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:25.994088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:46687)
0811 07:00:25.994260 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:25.994352 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 07:00:25.995448 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:25.995001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:54449)
0811 07:00:25.995152 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:25.995256 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 07:00:25.994591 1820 raft_consensus.cc:348] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
W20260811 07:00:25.995767 2014 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:00:25.995822 1820 raft_consensus.cc:729] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a3c12b81dbb40dc87114c3830531f93, State: Initialized, Role: FOLLOWER
I20260811 07:00:25.996060 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 1702
I20260811 07:00:25.996527 1820 consensus_queue.cc:260] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:25.996757 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:39733
--local_ip_for_outbound_sockets=127.4.40.2
--tserver_master_addrs=127.4.40.62:42515
--webserver_port=43469
--webserver_interface=127.4.40.2
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:00:25.998169 1820 ts_tablet_manager.cc:1428] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260811 07:00:25.998268 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:25.997824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:50097)
0811 07:00:25.997995 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:25.998084 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 07:00:25.999140 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:25.998859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:36167)
0811 07:00:25.998980 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:25.999037 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 07:00:26.348193 2022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:26.348474 2022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:26.348649 2022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:26.355676 2022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:26.355789 2022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:26.356019 2022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:00:26.365806 2022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:39733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=43469
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:26.367089 2022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:26.368523 2022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:26.382465 2032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:26.382426 2030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:26.382586 2028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:26.383653 2022 server_base.cc:1034] running on GCE node
I20260811 07:00:26.384409 2022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:26.385346 2022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:26.386570 2022 hybrid_clock.cc:648] HybridClock initialized: now 1786431626386491 us; error 43 us; skew 500 ppm
I20260811 07:00:26.386919 2022 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:26 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431626, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:26.390106 2022 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 07:00:26.391547 2022 webserver.cc:460] Webserver started at http://127.4.40.2:43469/ using document root <none> and password file <none>
I20260811 07:00:26.392329 2022 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:26.392484 2022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:26.397742 2022 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 07:00:26.400350 2045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:26.401624 2022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 07:00:26.401806 2022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "abaaa4db50f24a20be98945dd6778dd8"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "30a00aa6705eb3d36faa51ec609bdd5f"
server_key_iv: "cad61068c73226190d9356f5f2488af1"
server_key_version: "encryptionkey@0"
I20260811 07:00:26.402786 2022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:26.418783 2022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:26.422538 2022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:26.422822 2022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:26.423875 2022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:26.425246 2022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:26.425379 2022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:26.425509 2022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:26.425604 2022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:26.460309 2022 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:39733
I20260811 07:00:26.460366 2275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:39733 every 8 connection(s)
I20260811 07:00:26.461787 2022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:00:26.466396 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:26.464623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:38983)
0811 07:00:26.465373 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:26.465563 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":310,"threads_started":1}
W20260811 07:00:26.467083 2277 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:42515 (0 consecutive failures): Network error: Failed to ping master at 127.4.40.62:42515: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
I20260811 07:00:26.468009 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:26.467690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:49357)
0811 07:00:26.467827 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:26.467894 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 07:00:26.468892 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:26.468581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:52589)
0811 07:00:26.468717 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:26.468788 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 07:00:26.469142 2277 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:00:26.469677 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 2022
I20260811 07:00:26.470293 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:45139
--local_ip_for_outbound_sockets=127.4.40.3
--tserver_master_addrs=127.4.40.62:42515
--webserver_port=40639
--webserver_interface=127.4.40.3
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:26.815395 2290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:26.815732 2290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:26.815886 2290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:26.823096 2290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:26.823251 2290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:26.823447 2290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:00:26.832947 2290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:45139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=40639
--tserver_master_addrs=127.4.40.62:42515
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:26.834218 2290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:26.835572 2290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:26.849270 2299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:26.849200 2300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:26.849323 2303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:26.849993 2290 server_base.cc:1034] running on GCE node
I20260811 07:00:26.850533 2290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:26.851401 2290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:26.852640 2290 hybrid_clock.cc:648] HybridClock initialized: now 1786431626852595 us; error 54 us; skew 500 ppm
I20260811 07:00:26.852970 2290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:26 dist-test-slave-7vjv krb5kdc[32025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431626, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:26.855989 2290 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 07:00:26.857410 2290 webserver.cc:460] Webserver started at http://127.4.40.3:40639/ using document root <none> and password file <none>
I20260811 07:00:26.858076 2290 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:26.858227 2290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:26.863518 2290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:00:26.866159 2316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:26.867388 2290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:00:26.867560 2290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe"
format_stamp: "Formatted at 2026-08-11 06:59:40 on dist-test-slave-7vjv"
server_key: "a501381321d48676446ebf32b5c24388"
server_key_iv: "3643cc3a2d1a5bb377467cf7d2440cfc"
server_key_version: "encryptionkey@0"
I20260811 07:00:26.868649 2290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:26.879375 2290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:26.883186 2290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:26.883472 2290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:26.884495 2290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:26.885808 2290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:26.885921 2290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:26.886053 2290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:26.886145 2290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:26.919680 2290 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:45139
I20260811 07:00:26.919770 2530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:45139 every 8 connection(s)
I20260811 07:00:26.921211 2290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:00:26.923112 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 2290
I20260811 07:00:26.926595 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:26.924728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:34367)
0811 07:00:26.925559 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:26.925744 (+ 185us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":394,"threads_started":1}
W20260811 07:00:26.927186 2531 heartbeater.cc:644] Failed to heartbeat to 127.4.40.62:42515 (0 consecutive failures): Network error: Failed to ping master at 127.4.40.62:42515: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
I20260811 07:00:26.927980 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:26.927660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:36875)
0811 07:00:26.927817 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:26.927882 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 07:00:26.928836 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:26.928487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:37771)
0811 07:00:26.928638 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:26.928716 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 07:00:26.929080 2531 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 07:00:27.000834 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.000027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:47829)
0811 07:00:27.000543 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:27.000639 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":282,"threads_started":1}
I20260811 07:00:27.466208 2538 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:27.466439 2538 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:27.468017 2538 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
I20260811 07:00:27.470647 2544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.469665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:49557)
0811 07:00:27.470193 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:27.470320 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":308,"threads_started":1}
Aug 11 07:00:27 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
Aug 11 07:00:27 dist-test-slave-7vjv krb5kdc[32025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431625, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 07:00:27.494014 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.468521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:37215 (local address 127.4.40.2:39733)
0811 07:00:27.469722 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:27.469735 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:27.469764 (+ 29us) server_negotiation.cc:407] Connection header received
0811 07:00:27.472112 (+ 2348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:27.472122 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:27.472518 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:27.472871 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:27.475652 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.477567 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.480367 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.481374 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.481568 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:27.486856 (+ 5288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:27.486863 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:27.486875 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:27.486909 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:27.490097 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:27.490992 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:27.490999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:27.491004 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:27.491167 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:27.491686 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:27.491690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:27.491692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:27.492250 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:27.492448 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:27.493190 (+ 742us) server_negotiation.cc:299] Negotiation successful
0811 07:00:27.493599 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1025,"thread_start_us":869,"threads_started":1}
I20260811 07:00:27.494015 2542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.468784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:40563 (local address 127.4.40.3:45139)
0811 07:00:27.469697 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:27.469714 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:27.469756 (+ 42us) server_negotiation.cc:407] Connection header received
0811 07:00:27.472036 (+ 2280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:27.472048 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:27.472518 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:27.472866 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:27.475652 (+ 2786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.477567 (+ 1915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.480367 (+ 2800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.481374 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.481568 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:27.486748 (+ 5180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:27.486767 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:27.486782 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:27.486821 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:27.490045 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:27.490944 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:27.490953 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:27.490961 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:27.491130 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:27.491622 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:27.491627 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:27.491631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:27.492216 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:27.492422 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:27.493189 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 07:00:27.493581 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":997,"threads_started":1}
I20260811 07:00:27.494065 2539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.468353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:45139 (local address 127.4.40.1:40563)
0811 07:00:27.468989 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:27.469011 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:27.469121 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:27.473046 (+ 3925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:27.473058 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:27.474218 (+ 1160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:27.475423 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:27.475453 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.477804 (+ 2351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.477809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:27.480205 (+ 2396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:27.480222 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.481629 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.481636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:27.482105 (+ 469us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:27.483049 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:27.483094 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:27.486439 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:27.490197 (+ 3758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:27.490212 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:27.490229 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:27.490806 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:27.491263 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:27.491267 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:27.491270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:27.491495 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:27.492367 (+ 872us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:27.492374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:27.492696 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 07:00:27.492957 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 07:00:27.493545 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":446,"spinlock_wait_cycles":68224,"thread_start_us":643,"threads_started":1}
I20260811 07:00:27.494158 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.468131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:39733 (local address 127.4.40.1:37215)
0811 07:00:27.468356 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:27.468417 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:27.468942 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:27.473046 (+ 4104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:27.473058 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:27.474218 (+ 1160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:27.475423 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:27.475453 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.477806 (+ 2353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.477810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:27.480205 (+ 2395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:27.480222 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:27.481641 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:27.481648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:27.482105 (+ 457us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:27.483049 (+ 944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:27.483086 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:27.486603 (+ 3517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:27.490272 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:27.490276 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:27.490278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:27.490825 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:27.491351 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:27.491355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:27.491358 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:27.491537 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:27.492389 (+ 852us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:27.492394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:27.492696 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 07:00:27.492957 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 07:00:27.493702 (+ 745us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":775,"spinlock_wait_cycles":19968}
W20260811 07:00:27.497517 1814 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
W20260811 07:00:27.497773 1809 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
I20260811 07:00:27.497988 1814 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93; no voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe
I20260811 07:00:27.498445 2538 raft_consensus.cc:2726] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:00:27.930410 2546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:27.929555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:47573)
0811 07:00:27.930036 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:27.930158 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":242,"threads_started":1}
I20260811 07:00:28.002435 2550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:28.001648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:60135)
0811 07:00:28.002118 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:28.002218 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":230,"threads_started":1}
I20260811 07:00:28.472465 2551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:28.471692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:51067)
0811 07:00:28.472217 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:28.472298 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":271,"threads_started":1}
I20260811 07:00:28.932209 2552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:28.931400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:55011)
0811 07:00:28.931889 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:28.931983 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":279,"threads_started":1}
I20260811 07:00:29.004189 2555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:29.003321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:37105)
0811 07:00:29.003864 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:29.003941 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":249,"threads_started":1}
I20260811 07:00:29.189054 2556 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:29.189308 2556 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:29.190023 2556 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
W20260811 07:00:29.190999 1814 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
W20260811 07:00:29.191146 1809 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
I20260811 07:00:29.191316 1809 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93; no voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe
I20260811 07:00:29.191617 2556 raft_consensus.cc:2726] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:00:29.473980 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:29.473320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:51539)
0811 07:00:29.473732 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:29.473808 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":213,"threads_started":1}
I20260811 07:00:29.933751 2562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:29.933021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:44081)
0811 07:00:29.933493 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:29.933582 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":256,"threads_started":1}
I20260811 07:00:30.005641 2564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:30.004990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:37907)
0811 07:00:30.005390 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:30.005469 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":204,"threads_started":1}
I20260811 07:00:30.475551 2569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:30.474799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:46105)
0811 07:00:30.475249 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:30.475335 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":231,"threads_started":1}
I20260811 07:00:30.935384 2571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:30.934665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:50799)
0811 07:00:30.935131 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:30.935207 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
I20260811 07:00:31.007635 2572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:31.006846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:57715)
0811 07:00:31.007343 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:31.007424 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394,"thread_start_us":232,"threads_started":1}
I20260811 07:00:31.477221 2573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:31.476544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:34475)
0811 07:00:31.476972 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:31.477048 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":231,"threads_started":1}
I20260811 07:00:31.937285 2575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:31.936440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:39067)
0811 07:00:31.936952 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:31.937114 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":284,"threads_started":1}
I20260811 07:00:32.009339 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:32.008647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:34453)
0811 07:00:32.009072 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:32.009171 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":211,"threads_started":1}
I20260811 07:00:32.478727 2577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:32.477993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:53423)
0811 07:00:32.478449 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:32.478541 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":245,"threads_started":1}
I20260811 07:00:32.482031 2579 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:32.482282 2579 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:32.483035 2579 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
W20260811 07:00:32.483982 1809 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
W20260811 07:00:32.484162 1814 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
I20260811 07:00:32.484330 1814 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93; no voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe
I20260811 07:00:32.484632 2579 raft_consensus.cc:2726] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:00:32.938928 2580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:32.938187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:44319)
0811 07:00:32.938663 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:32.938743 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":261,"threads_started":1}
I20260811 07:00:33.011204 2583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:33.010402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:48919)
0811 07:00:33.010888 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:33.010981 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":242,"threads_started":1}
I20260811 07:00:33.480541 2585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:33.479663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:44203)
0811 07:00:33.480245 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:33.480326 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":246,"threads_started":1}
I20260811 07:00:33.940730 2587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:33.939888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:60319)
0811 07:00:33.940455 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:33.940546 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":297,"threads_started":1}
I20260811 07:00:34.012869 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:34.012154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:40925)
0811 07:00:34.012616 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:34.012710 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":242,"threads_started":1}
I20260811 07:00:34.482090 2592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:34.481426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:34161)
0811 07:00:34.481818 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:34.481903 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":203,"threads_started":1}
I20260811 07:00:34.787191 2593 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:34.787376 2593 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:34.787937 2593 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
W20260811 07:00:34.789041 1809 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
W20260811 07:00:34.789247 1814 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
I20260811 07:00:34.789371 1814 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93; no voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe
I20260811 07:00:34.789604 2593 raft_consensus.cc:2726] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:00:34.942499 2594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:34.941746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:36779)
0811 07:00:34.942182 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:34.942294 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":227,"threads_started":1}
I20260811 07:00:35.014786 2595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:35.013814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:37451)
0811 07:00:35.014454 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:35.014559 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":296,"threads_started":1}
I20260811 07:00:35.483837 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:35.482984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:50607)
0811 07:00:35.483568 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:35.483668 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":321,"threads_started":1}
I20260811 07:00:35.944347 2601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:35.943487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:42441)
0811 07:00:35.944041 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:35.944186 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":297,"threads_started":1}
I20260811 07:00:36.016551 2603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:36.015803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:37467)
0811 07:00:36.016289 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:36.016372 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":242,"threads_started":1}
I20260811 07:00:36.485430 2606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:36.484759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:60417)
0811 07:00:36.485174 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:36.485250 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":215,"threads_started":1}
I20260811 07:00:36.945858 2607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:36.945194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:60407)
0811 07:00:36.945602 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:36.945676 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":224,"threads_started":1}
I20260811 07:00:37.018092 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:37.017413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:33985)
0811 07:00:37.017838 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:37.017916 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":233,"threads_started":1}
I20260811 07:00:37.487145 2609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:37.486302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:52433)
0811 07:00:37.486890 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:37.486970 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":345,"threads_started":1}
I20260811 07:00:37.947533 2611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:37.946701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:38449)
0811 07:00:37.947246 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:37.947339 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":286,"threads_started":1}
I20260811 07:00:38.019630 2612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:38.018916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:51319)
0811 07:00:38.019362 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:38.019440 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":235,"threads_started":1}
I20260811 07:00:38.488829 2613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:38.488041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:45395)
0811 07:00:38.488568 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:38.488650 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":286,"threads_started":1}
I20260811 07:00:38.949213 2614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:38.948501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:59449)
0811 07:00:38.948947 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:38.949030 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260811 07:00:39.021289 2616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:39.020536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:59383)
0811 07:00:39.020994 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:39.021096 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":250,"threads_started":1}
I20260811 07:00:39.490597 2617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:39.489744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:41883)
0811 07:00:39.490328 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:39.490411 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":347,"threads_started":1}
I20260811 07:00:39.950831 2621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:39.950128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:38609)
0811 07:00:39.950600 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:39.950671 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":241,"threads_started":1}
I20260811 07:00:40.023149 2622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:40.022304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:32963)
0811 07:00:40.022855 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:40.022932 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":336,"threads_started":1}
I20260811 07:00:40.492424 2625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:40.491643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:57671)
0811 07:00:40.492132 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:40.492222 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":221,"threads_started":1}
I20260811 07:00:40.952629 2629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:40.951790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:59961)
0811 07:00:40.952337 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:40.952416 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":318,"threads_started":1}
I20260811 07:00:41.025094 2634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:41.024369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:57491)
0811 07:00:41.024838 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:41.024915 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":229,"threads_started":1}
I20260811 07:00:41.494184 2635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:41.493493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.2:41221)
0811 07:00:41.493941 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:41.494026 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":231,"threads_started":1}
I20260811 07:00:41.927915 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 1702
I20260811 07:00:41.942765 2636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:41.928986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.4.40.1:37389)
0811 07:00:41.929615 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:41.929627 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:41.929661 (+ 34us) server_negotiation.cc:407] Connection header received
0811 07:00:41.929745 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:41.929757 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:41.930079 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:41.930278 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:41.931962 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:41.933213 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:41.934422 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:41.934965 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:41.935075 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:41.937662 (+ 2587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:41.937673 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:41.937685 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:41.937715 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:41.940425 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:41.941048 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:41.941054 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:41.941056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:41.941187 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:41.941521 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:41.941525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:41.941526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:41.941868 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:41.942098 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:41.942322 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 07:00:41.942514 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":257,"threads_started":1}
I20260811 07:00:41.944083 2002 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:41.954437 2638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:41.953764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.3:41475)
0811 07:00:41.954175 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:41.954277 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":215,"threads_started":1}
I20260811 07:00:42.036607 2639 raft_consensus.cc:482] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:42.036957 2639 raft_consensus.cc:504] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "abaaa4db50f24a20be98945dd6778dd8" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 39733 } } peers { permanent_uuid: "3a3c12b81dbb40dc87114c3830531f93" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37389 } } peers { permanent_uuid: "e56cff34d6f04b7b8b2e69ce3745cfbe" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 45139 } }
I20260811 07:00:42.037412 2643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:42.032773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:42515 (local address 127.4.40.1:57071)
0811 07:00:42.036859 (+ 4086us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:42.037038 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.62:42515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3935,"thread_start_us":3780,"threads_started":1}
I20260811 07:00:42.037881 2639 leader_election.cc:288] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733), e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139)
W20260811 07:00:42.038712 1809 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer abaaa4db50f24a20be98945dd6778dd8 (127.4.40.2:39733): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
W20260811 07:00:42.038854 1814 leader_election.cc:341] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e56cff34d6f04b7b8b2e69ce3745cfbe (127.4.40.3:45139): Not found: Tablet not found: 7b011940b294421eb4cf194c098484d3
I20260811 07:00:42.038977 1814 leader_election.cc:302] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 3a3c12b81dbb40dc87114c3830531f93; no voters: abaaa4db50f24a20be98945dd6778dd8, e56cff34d6f04b7b8b2e69ce3745cfbe
I20260811 07:00:42.039340 2639 raft_consensus.cc:2726] T 7b011940b294421eb4cf194c098484d3 P 3a3c12b81dbb40dc87114c3830531f93 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 07:00:42.042092 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 1702
I20260811 07:00:42.075367 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 2022
I20260811 07:00:42.089617 2644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:42.076219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32962 (local address 127.4.40.2:39733)
0811 07:00:42.076722 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:42.076727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:42.076742 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:00:42.076797 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:42.076801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:42.076858 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:42.076976 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:42.078443 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:42.079485 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:42.080971 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:42.081412 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:42.081552 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:42.084273 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:42.084278 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:42.084282 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:42.084325 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:42.086837 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:42.087517 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:42.087530 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:42.087532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:42.087658 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:42.088045 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:42.088049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:42.088051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:42.088417 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:42.088705 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:42.088991 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 07:00:42.089214 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":272,"threads_started":1}
I20260811 07:00:42.090454 2268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:42.175730 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 2022
I20260811 07:00:42.204200 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 2290
I20260811 07:00:42.217698 2647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:42.205093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36328 (local address 127.4.40.3:45139)
0811 07:00:42.205626 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:42.205630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:42.205648 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:00:42.205703 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:42.205707 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:42.205772 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:42.205863 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:42.207264 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:42.208061 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:42.209415 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:42.209837 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:42.209937 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:42.212625 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:42.212630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:42.212633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:42.212666 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:42.215237 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:42.215901 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:42.215905 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:42.215907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:42.216043 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:42.216552 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:42.216555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:42.216557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:42.216807 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:42.217030 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:42.217232 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 07:00:42.217398 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":275,"threads_started":1}
I20260811 07:00:42.218468 2522 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:42.304123 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 2290
2026-08-11T07:00:42Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65223 ms)
[----------] 30 tests from SecurityITest (308415 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:00:42 dist-test-slave-7vjv krb5kdc[2658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:00:42 dist-test-slave-7vjv krb5kdc[2658](info): set up 2 sockets
Aug 11 07:00:42 dist-test-slave-7vjv krb5kdc[2658](info): commencing operation
krb5kdc: starting...
W20260811 07:00:43.698118 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:00:43 dist-test-slave-7vjv krb5kdc[2658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:00:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:00:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:00:44.020668 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46533
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:46533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:44.399781 2684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:44.400077 2684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:44.400287 2684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:44.407409 2684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:00:44.407507 2684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:44.407552 2684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:44.407635 2684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:00:44.407718 2684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:00:44.418165 2684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:46533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:46533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:44.419467 2684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:44.420915 2684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:44.432825 2692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:44.432909 2696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:44.432863 2693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:44.437145 2684 server_base.cc:1034] running on GCE node
I20260811 07:00:44.437815 2684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:44.439065 2684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:44.440282 2684 hybrid_clock.cc:648] HybridClock initialized: now 1786431644440242 us; error 46 us; skew 500 ppm
I20260811 07:00:44.440611 2684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:44 dist-test-slave-7vjv krb5kdc[2658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431644, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:44.443801 2684 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:00:44.445364 2684 webserver.cc:460] Webserver started at http://127.4.40.62:41551/ using document root <none> and password file <none>
I20260811 07:00:44.446012 2684 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:44.446110 2684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:44.446358 2684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:44.449294 2684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "84780e205f534829ac99dbc92121310f"
format_stamp: "Formatted at 2026-08-11 07:00:44 on dist-test-slave-7vjv"
server_key: "67dea9cee190890ece105f9b7c9199e6"
server_key_iv: "da4f83db22d969e84a3250feb1070d94"
server_key_version: "encryptionkey@0"
I20260811 07:00:44.449977 2684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "84780e205f534829ac99dbc92121310f"
format_stamp: "Formatted at 2026-08-11 07:00:44 on dist-test-slave-7vjv"
server_key: "67dea9cee190890ece105f9b7c9199e6"
server_key_iv: "da4f83db22d969e84a3250feb1070d94"
server_key_version: "encryptionkey@0"
I20260811 07:00:44.455065 2684 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 07:00:44.458261 2715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:44.459445 2684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:00:44.459649 2684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "84780e205f534829ac99dbc92121310f"
format_stamp: "Formatted at 2026-08-11 07:00:44 on dist-test-slave-7vjv"
server_key: "67dea9cee190890ece105f9b7c9199e6"
server_key_iv: "da4f83db22d969e84a3250feb1070d94"
server_key_version: "encryptionkey@0"
I20260811 07:00:44.459841 2684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:44.476487 2684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:44.480281 2684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:44.480566 2684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:44.498714 2684 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:46533
I20260811 07:00:44.498780 2806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:46533 every 8 connection(s)
I20260811 07:00:44.500084 2684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:00:44.503207 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 2684
I20260811 07:00:44.503459 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:00:44.504282 2807 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:44.504426 4256 external_mini_cluster.cc:1428] Setting key 4df483e4cbbaa324e43a75b156bbb3cc
I20260811 07:00:44.519649 2807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f: Bootstrap starting.
Aug 11 07:00:44 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:44.522948 2807 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:44.524001 2807 log.cc:824] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:44.526947 2807 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f: No bootstrap required, opened a new log
I20260811 07:00:44.528738 2810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:44.506573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.4.40.62:46533)
0811 07:00:44.507446 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:44.507457 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:44.507501 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:00:44.509426 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:44.509446 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:44.509817 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:44.510167 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:44.512066 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:44.513338 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:44.515402 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:44.516181 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:44.516318 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:44.521366 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:44.521374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:44.521391 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:44.521426 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:44.524577 (+ 3151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:44.525274 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:44.525289 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:44.525295 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:44.525479 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:44.525904 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:44.525909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:44.525912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:44.526470 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:44.526768 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:44.527262 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 07:00:44.527708 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":426,"threads_started":1}
I20260811 07:00:44.535044 2807 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84780e205f534829ac99dbc92121310f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46533 } }
I20260811 07:00:44.535452 2807 raft_consensus.cc:374] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:44.535559 2807 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84780e205f534829ac99dbc92121310f, State: Initialized, Role: FOLLOWER
I20260811 07:00:44.536144 2807 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84780e205f534829ac99dbc92121310f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46533 } }
I20260811 07:00:44.536404 2807 raft_consensus.cc:388] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:00:44.536569 2807 raft_consensus.cc:482] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:00:44.536761 2807 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:44.538932 2807 raft_consensus.cc:504] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84780e205f534829ac99dbc92121310f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46533 } }
I20260811 07:00:44.539450 2807 leader_election.cc:302] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 84780e205f534829ac99dbc92121310f; no voters:
I20260811 07:00:44.540230 2807 leader_election.cc:288] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:00:44.540361 2813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:44.541039 2813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [term 1 LEADER]: Becoming Leader. State: Replica: 84780e205f534829ac99dbc92121310f, State: Running, Role: LEADER
I20260811 07:00:44.541574 2813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84780e205f534829ac99dbc92121310f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46533 } }
I20260811 07:00:44.542289 2807 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:00:44.544248 2816 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "84780e205f534829ac99dbc92121310f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84780e205f534829ac99dbc92121310f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46533 } } }
I20260811 07:00:44.544691 2816 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:44.544821 2817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 84780e205f534829ac99dbc92121310f. Latest consensus state: current_term: 1 leader_uuid: "84780e205f534829ac99dbc92121310f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "84780e205f534829ac99dbc92121310f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 46533 } } }
I20260811 07:00:44.545161 2817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:44.548959 2824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:00:44.554329 2824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:00:44.563073 2824 catalog_manager.cc:1347] Generated new cluster ID: 11df5baa4fbe4bceaabff7ac73c379f1
I20260811 07:00:44.563201 2824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:00:44.577746 2824 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:00:44.579011 2824 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:00:44.586891 2824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f: Generated new TSK 0
I20260811 07:00:44.587718 2824 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 07:00:44.668352 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46533
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:45.015681 2847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:45.016044 2847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:45.016238 2847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:45.023452 2847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:45.023599 2847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:45.023783 2847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:00:45.033785 2847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:45.035105 2847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:45.036468 2847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:45.050114 2858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:45.050180 2861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:45.050132 2859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:45.051177 2847 server_base.cc:1034] running on GCE node
I20260811 07:00:45.051754 2847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:45.052722 2847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:45.053920 2847 hybrid_clock.cc:648] HybridClock initialized: now 1786431645053877 us; error 32 us; skew 500 ppm
I20260811 07:00:45.054200 2847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:45 dist-test-slave-7vjv krb5kdc[2658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431645, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:45.057396 2847 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 07:00:45.058772 2847 webserver.cc:460] Webserver started at http://127.4.40.1:34085/ using document root <none> and password file <none>
I20260811 07:00:45.059446 2847 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:45.059593 2847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:45.059903 2847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:45.062896 2847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "97909824241b492ea65d0366a55eb2f7"
format_stamp: "Formatted at 2026-08-11 07:00:45 on dist-test-slave-7vjv"
server_key: "11b8934dd66514806ae7925cac460c21"
server_key_iv: "9cf584d9f85bcf051d466d3526607558"
server_key_version: "encryptionkey@0"
I20260811 07:00:45.063722 2847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "97909824241b492ea65d0366a55eb2f7"
format_stamp: "Formatted at 2026-08-11 07:00:45 on dist-test-slave-7vjv"
server_key: "11b8934dd66514806ae7925cac460c21"
server_key_iv: "9cf584d9f85bcf051d466d3526607558"
server_key_version: "encryptionkey@0"
I20260811 07:00:45.068933 2847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 07:00:45.072345 2871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:45.073606 2847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:00:45.073778 2847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "97909824241b492ea65d0366a55eb2f7"
format_stamp: "Formatted at 2026-08-11 07:00:45 on dist-test-slave-7vjv"
server_key: "11b8934dd66514806ae7925cac460c21"
server_key_iv: "9cf584d9f85bcf051d466d3526607558"
server_key_version: "encryptionkey@0"
I20260811 07:00:45.073980 2847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:45.092115 2847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:45.096031 2847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:45.096382 2847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:45.097352 2847 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:45.098656 2847 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:45.098757 2847 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:45.098904 2847 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:45.098965 2847 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:45.134439 2847 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:38791
I20260811 07:00:45.134470 3082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:38791 every 8 connection(s)
I20260811 07:00:45.135931 2847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:00:45.142341 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 2847
I20260811 07:00:45.142613 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 07:00:45.143373 4256 external_mini_cluster.cc:1428] Setting key 3b92b967fc4f3eaa40cdb876866c260b
Aug 11 07:00:45 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431645, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:45.156188 3086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:45.138642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57571 (local address 127.4.40.62:46533)
0811 07:00:45.139138 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:45.139142 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:45.140050 (+ 908us) server_negotiation.cc:407] Connection header received
0811 07:00:45.141901 (+ 1851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:45.141910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:45.142027 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:45.142165 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:45.143913 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.144844 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.146556 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.146996 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.147140 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:45.150627 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:45.150632 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:45.150635 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:45.150667 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:45.153207 (+ 2540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:45.153901 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:45.153909 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:45.153912 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:45.154037 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:45.154453 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:45.154459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:45.154463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:45.154733 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:45.154891 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:45.155579 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 07:00:45.155799 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":255,"threads_started":1}
I20260811 07:00:45.157107 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:45.139180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46533 (local address 127.4.40.1:57571)
0811 07:00:45.139890 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:45.139928 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:45.141634 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:45.142386 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:45.142400 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:45.143054 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:45.143741 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:45.143760 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.145024 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.145028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:45.146390 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:45.146403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.147236 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.147244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:45.147480 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:45.148232 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:45.148259 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:45.150413 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:45.153342 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:45.153350 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:45.153370 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:45.153783 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:45.154151 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:45.154156 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:45.154160 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:45.154340 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:45.154875 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:45.154882 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:45.155205 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 07:00:45.155466 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 07:00:45.155961 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":276,"threads_started":1}
I20260811 07:00:45.158643 3084 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46533
I20260811 07:00:45.158962 3084 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:45.159858 3084 heartbeater.cc:511] Master 127.4.40.62:46533 requested a full tablet report, sending...
I20260811 07:00:45.162024 2748 ts_manager.cc:194] Registered new tserver with Master: 97909824241b492ea65d0366a55eb2f7 (127.4.40.1:38791)
I20260811 07:00:45.163552 2748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:57571
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 07:00:45.216571 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46533
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:45.565577 3097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:45.565867 3097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:45.566006 3097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:45.572849 3097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:45.572961 3097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:45.573137 3097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:00:45.583009 3097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:45.584280 3097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:45.585666 3097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:45.599208 3106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:45.599236 3107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:45.599341 3112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:45.600028 3097 server_base.cc:1034] running on GCE node
I20260811 07:00:45.600687 3097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:45.601586 3097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:45.602785 3097 hybrid_clock.cc:648] HybridClock initialized: now 1786431645602739 us; error 40 us; skew 500 ppm
I20260811 07:00:45.603101 3097 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:45 dist-test-slave-7vjv krb5kdc[2658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431645, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:45.609871 3097 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 07:00:45.611222 3097 webserver.cc:460] Webserver started at http://127.4.40.2:40387/ using document root <none> and password file <none>
I20260811 07:00:45.611930 3097 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:45.612071 3097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:45.612427 3097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:45.615278 3097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "62d6fd146a2b4a94907c1a9736180e6d"
format_stamp: "Formatted at 2026-08-11 07:00:45 on dist-test-slave-7vjv"
server_key: "b18a269db494cc2ce1077a12528a787b"
server_key_iv: "a3bcf17149aa2441dcba7b1e80f867ab"
server_key_version: "encryptionkey@0"
I20260811 07:00:45.616050 3097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "62d6fd146a2b4a94907c1a9736180e6d"
format_stamp: "Formatted at 2026-08-11 07:00:45 on dist-test-slave-7vjv"
server_key: "b18a269db494cc2ce1077a12528a787b"
server_key_iv: "a3bcf17149aa2441dcba7b1e80f867ab"
server_key_version: "encryptionkey@0"
I20260811 07:00:45.621102 3097 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 07:00:45.624469 3126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:45.625730 3097 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 07:00:45.625923 3097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "62d6fd146a2b4a94907c1a9736180e6d"
format_stamp: "Formatted at 2026-08-11 07:00:45 on dist-test-slave-7vjv"
server_key: "b18a269db494cc2ce1077a12528a787b"
server_key_iv: "a3bcf17149aa2441dcba7b1e80f867ab"
server_key_version: "encryptionkey@0"
I20260811 07:00:45.626128 3097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:45.650316 3097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:45.655118 3097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:45.655418 3097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:45.656436 3097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:45.657788 3097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:45.657874 3097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:45.658061 3097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:45.658114 3097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:45.692227 3097 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:34821
I20260811 07:00:45.692339 3344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:34821 every 8 connection(s)
I20260811 07:00:45.693555 3097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:00:45.697609 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 3097
I20260811 07:00:45.697842 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 07:00:45.698441 4256 external_mini_cluster.cc:1428] Setting key 9ba00cb79ebee606cb2d503878a05251
Aug 11 07:00:45 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431645, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:45.715678 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:45.696186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:52515 (local address 127.4.40.62:46533)
0811 07:00:45.696620 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:45.696625 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:45.697802 (+ 1177us) server_negotiation.cc:407] Connection header received
0811 07:00:45.700224 (+ 2422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:45.700229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:45.700290 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:45.700404 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:45.701981 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.703304 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.704974 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.705726 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.705863 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:45.709730 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:45.709735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:45.709738 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:45.709799 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:45.712451 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:45.713366 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:45.713370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:45.713372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:45.713498 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:45.713979 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:45.713982 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:45.713984 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:45.714232 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:45.714341 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:45.715253 (+ 912us) server_negotiation.cc:299] Negotiation successful
0811 07:00:45.715410 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":193,"threads_started":1}
I20260811 07:00:45.716868 3349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:45.696727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46533 (local address 127.4.40.2:52515)
0811 07:00:45.697600 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:45.697653 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:45.699899 (+ 2246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:45.700676 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:45.700686 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:45.701259 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:45.701806 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:45.701822 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.703448 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.703451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:45.704802 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:45.704814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:45.705936 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:45.705941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:45.706140 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:45.706824 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:45.706845 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:45.709507 (+ 2662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:45.712582 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:45.712592 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:45.712609 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:45.713235 (+ 626us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:45.713612 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:45.713619 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:45.713625 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:45.713869 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:45.714358 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:45.714368 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:45.714844 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 07:00:45.715169 (+ 325us) client_negotiation.cc:240] Negotiation successful
0811 07:00:45.715733 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":313,"threads_started":1}
I20260811 07:00:45.718225 3346 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46533
I20260811 07:00:45.718564 3346 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:45.719395 3346 heartbeater.cc:511] Master 127.4.40.62:46533 requested a full tablet report, sending...
I20260811 07:00:45.721084 2748 ts_manager.cc:194] Registered new tserver with Master: 62d6fd146a2b4a94907c1a9736180e6d (127.4.40.2:34821)
I20260811 07:00:45.721856 2748 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:52515
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 07:00:45.771575 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46533
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:39261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:46.120868 3356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:46.121129 3356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:46.121316 3356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:46.129827 3356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:46.129935 3356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:46.130107 3356 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:00:46.139890 3356 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:39261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:46533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:46.141057 3356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:46.142691 3356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:46.157123 3364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:46.157255 3367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:46.157255 3365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:46.158041 3356 server_base.cc:1034] running on GCE node
I20260811 07:00:46.158591 3356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:46.159526 3356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:46.160773 3356 hybrid_clock.cc:648] HybridClock initialized: now 1786431646160721 us; error 50 us; skew 500 ppm
I20260811 07:00:46.161074 3356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:46 dist-test-slave-7vjv krb5kdc[2658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:46.164333 3356 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 07:00:46.165709 3356 webserver.cc:460] Webserver started at http://127.4.40.3:39493/ using document root <none> and password file <none>
I20260811 07:00:46.166419 3356 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:46.166523 3356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:46.166796 3356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:46.167164 3084 heartbeater.cc:503] Master 127.4.40.62:46533 was elected leader, sending a full tablet report...
I20260811 07:00:46.169885 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "31be710672c3434ba87eba80860734d4"
format_stamp: "Formatted at 2026-08-11 07:00:46 on dist-test-slave-7vjv"
server_key: "835cb37812e56804b1816d5d75fe1c30"
server_key_iv: "cdfe35334e59e8b40127c619369d2fa4"
server_key_version: "encryptionkey@0"
I20260811 07:00:46.170710 3356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "31be710672c3434ba87eba80860734d4"
format_stamp: "Formatted at 2026-08-11 07:00:46 on dist-test-slave-7vjv"
server_key: "835cb37812e56804b1816d5d75fe1c30"
server_key_iv: "cdfe35334e59e8b40127c619369d2fa4"
server_key_version: "encryptionkey@0"
I20260811 07:00:46.175580 3356 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 07:00:46.178952 3389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:46.180299 3356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:46.180474 3356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "31be710672c3434ba87eba80860734d4"
format_stamp: "Formatted at 2026-08-11 07:00:46 on dist-test-slave-7vjv"
server_key: "835cb37812e56804b1816d5d75fe1c30"
server_key_iv: "cdfe35334e59e8b40127c619369d2fa4"
server_key_version: "encryptionkey@0"
I20260811 07:00:46.180707 3356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:46.201963 3356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:46.206030 3356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:46.206327 3356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:46.207304 3356 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:46.208702 3356 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:46.208814 3356 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:46.208952 3356 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:46.209043 3356 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:46.243126 3356 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:44699
I20260811 07:00:46.243177 3602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:44699 every 8 connection(s)
I20260811 07:00:46.244685 3356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:00:46.251919 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 3356
I20260811 07:00:46.252193 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 07:00:46.252650 4256 external_mini_cluster.cc:1428] Setting key a976995238cf422e9bab47775fd4361a
Aug 11 07:00:46 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431646, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:46.264968 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.247454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:54641 (local address 127.4.40.62:46533)
0811 07:00:46.247964 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.247968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.248832 (+ 864us) server_negotiation.cc:407] Connection header received
0811 07:00:46.250646 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.250651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.250753 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.250887 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:46.252521 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.253452 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.255225 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.255717 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.255872 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.259197 (+ 3325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:46.259203 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:46.259206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:46.259236 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:46.261828 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.262591 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.262596 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.262598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.262713 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.263158 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.263163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.263166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.263449 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:46.263634 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.264355 (+ 721us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.264585 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":253,"threads_started":1}
I20260811 07:00:46.265935 3615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.247900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:46533 (local address 127.4.40.3:54641)
0811 07:00:46.248684 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:46.248723 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:46.250358 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:46.251106 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:46.251114 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:46.251673 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:46.252328 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.252372 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.253620 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.253625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.255087 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.255100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.255968 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.255976 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.256258 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.256867 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:46.256889 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:46.259000 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:46.262006 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:46.262015 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:46.262029 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:46.262436 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:46.262830 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:46.262835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:46.262839 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:46.263041 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:46.263608 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:46.263616 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:46.263931 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 07:00:46.264234 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 07:00:46.264825 (+ 591us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":332,"threads_started":1}
I20260811 07:00:46.267338 3603 heartbeater.cc:347] Connected to a master server at 127.4.40.62:46533
I20260811 07:00:46.267658 3603 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:46.268502 3603 heartbeater.cc:511] Master 127.4.40.62:46533 requested a full tablet report, sending...
I20260811 07:00:46.269914 2744 ts_manager.cc:194] Registered new tserver with Master: 31be710672c3434ba87eba80860734d4 (127.4.40.3:44699)
I20260811 07:00:46.270792 2744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:54641
I20260811 07:00:46.279930 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:00:46.297109 3609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.284471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44932 (local address 127.4.40.62:46533)
0811 07:00:46.284690 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.284697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.285086 (+ 389us) server_negotiation.cc:407] Connection header received
0811 07:00:46.285312 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.285317 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.285384 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.285480 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:46.286919 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.287602 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.288953 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.289354 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.289498 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.291859 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:46.291865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:46.291869 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:46.291903 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:46.294366 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.295074 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.295081 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.295084 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.295205 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.295587 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.295592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.295596 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.295920 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:46.296240 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.296567 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.296791 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 07:00:46.301069 2744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:00:46.307920 2744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:00:46.329952 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.319491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37422 (local address 127.4.40.1:38791)
0811 07:00:46.320228 (+ 737us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.320239 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.321002 (+ 763us) server_negotiation.cc:407] Connection header received
0811 07:00:46.321425 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.321436 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.321730 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.321924 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:46.323323 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.325038 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.327390 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.328715 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.328851 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.329119 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.329415 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.329609 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":355,"threads_started":1}
I20260811 07:00:46.330099 3636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.319307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:38791 (local address 127.0.0.1:37422)
0811 07:00:46.320842 (+ 1535us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:46.320881 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:46.321168 (+ 287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:46.322342 (+ 1174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:46.322349 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:46.322419 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:46.323096 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.323113 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.325202 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.325208 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.327192 (+ 1984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.327206 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.328903 (+ 1697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.328909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.329124 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.329202 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 07:00:46.329349 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 07:00:46.329553 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1259,"mutex_wait_us":49,"spinlock_wait_cycles":6400,"thread_start_us":1094,"threads_started":1}
I20260811 07:00:46.331940 3637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.320184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:34821 (local address 127.0.0.1:51568)
0811 07:00:46.320934 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:46.320952 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:46.321166 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:46.322336 (+ 1170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:46.322346 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:46.322404 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:46.323674 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.323688 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.325553 (+ 1865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.325559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.327814 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.327829 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.331021 (+ 3192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.331031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.331231 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.331268 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 07:00:46.331390 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 07:00:46.331518 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":624,"mutex_wait_us":598,"spinlock_wait_cycles":2304,"thread_start_us":477,"threads_started":1}
I20260811 07:00:46.332033 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.319833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.4.40.2:34821)
0811 07:00:46.320493 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.320501 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.321051 (+ 550us) server_negotiation.cc:407] Connection header received
0811 07:00:46.321420 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.321429 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.321683 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.321857 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:46.323871 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.325394 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.327988 (+ 2594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.329565 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.329682 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.329876 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.331481 (+ 1605us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.331722 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":597,"threads_started":1}
I20260811 07:00:46.332207 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.321661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54150 (local address 127.4.40.3:44699)
0811 07:00:46.323291 (+ 1630us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.323298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.324044 (+ 746us) server_negotiation.cc:407] Connection header received
0811 07:00:46.324169 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.324178 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.324411 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.324597 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:46.325187 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.326769 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.328931 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.330393 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.330526 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.330780 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.331643 (+ 863us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.331816 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1461,"thread_start_us":1302,"threads_started":1}
I20260811 07:00:46.332628 3640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.321280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44699 (local address 127.0.0.1:54150)
0811 07:00:46.323767 (+ 2487us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:46.323785 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:46.323912 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:46.324593 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:46.324597 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:46.324621 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:46.324998 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.325008 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.326940 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.326945 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.328718 (+ 1773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.328736 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.330957 (+ 2221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.330965 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.331154 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.331186 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 07:00:46.331286 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 07:00:46.331405 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2346,"thread_start_us":2180,"threads_started":1}
I20260811 07:00:46.335141 2961 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0d6bfb5b149cd87e83471fe05eccd (DEFAULT_TABLE table=test-table [id=f6f64e30c65b4f65a3c4adcc3430af2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:46.336808 2961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0d6bfb5b149cd87e83471fe05eccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:46.336601 3216 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0d6bfb5b149cd87e83471fe05eccd (DEFAULT_TABLE table=test-table [id=f6f64e30c65b4f65a3c4adcc3430af2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:46.337922 3216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0d6bfb5b149cd87e83471fe05eccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:46.337574 3485 tablet_service.cc:1467] Processing CreateTablet for tablet e9f0d6bfb5b149cd87e83471fe05eccd (DEFAULT_TABLE table=test-table [id=f6f64e30c65b4f65a3c4adcc3430af2f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:46.338944 3485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e9f0d6bfb5b149cd87e83471fe05eccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:46.353335 3643 tablet_bootstrap.cc:492] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Bootstrap starting.
I20260811 07:00:46.356665 3643 tablet_bootstrap.cc:654] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:46.357688 3642 tablet_bootstrap.cc:492] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Bootstrap starting.
I20260811 07:00:46.357918 3643 log.cc:824] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:46.362910 3642 tablet_bootstrap.cc:654] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:46.364638 3642 log.cc:824] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:46.366753 3643 tablet_bootstrap.cc:492] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: No bootstrap required, opened a new log
I20260811 07:00:46.367122 3643 ts_tablet_manager.cc:1397] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260811 07:00:46.367964 3645 tablet_bootstrap.cc:492] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Bootstrap starting.
I20260811 07:00:46.371003 3642 tablet_bootstrap.cc:492] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: No bootstrap required, opened a new log
I20260811 07:00:46.371263 3642 ts_tablet_manager.cc:1397] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.005s
I20260811 07:00:46.375183 3645 tablet_bootstrap.cc:654] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:46.376456 3645 log.cc:824] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:46.376492 3643 raft_consensus.cc:348] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.376935 3643 raft_consensus.cc:374] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:46.377117 3643 raft_consensus.cc:729] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62d6fd146a2b4a94907c1a9736180e6d, State: Initialized, Role: FOLLOWER
I20260811 07:00:46.377862 3643 consensus_queue.cc:260] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.379962 3645 tablet_bootstrap.cc:492] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: No bootstrap required, opened a new log
I20260811 07:00:46.380319 3645 ts_tablet_manager.cc:1397] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.000s
I20260811 07:00:46.380856 3346 heartbeater.cc:503] Master 127.4.40.62:46533 was elected leader, sending a full tablet report...
I20260811 07:00:46.381579 3643 ts_tablet_manager.cc:1428] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
I20260811 07:00:46.381301 3642 raft_consensus.cc:348] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.381718 3642 raft_consensus.cc:374] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:46.381880 3642 raft_consensus.cc:729] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97909824241b492ea65d0366a55eb2f7, State: Initialized, Role: FOLLOWER
I20260811 07:00:46.382499 3642 consensus_queue.cc:260] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.384459 3642 ts_tablet_manager.cc:1428] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
W20260811 07:00:46.388716 3085 tablet.cc:2369] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:46.389528 3645 raft_consensus.cc:348] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.389878 3645 raft_consensus.cc:374] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:46.389981 3645 raft_consensus.cc:729] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31be710672c3434ba87eba80860734d4, State: Initialized, Role: FOLLOWER
I20260811 07:00:46.390501 3645 consensus_queue.cc:260] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.391090 3651 raft_consensus.cc:482] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:46.391319 3651 raft_consensus.cc:504] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.392151 3603 heartbeater.cc:503] Master 127.4.40.62:46533 was elected leader, sending a full tablet report...
I20260811 07:00:46.392593 3645 ts_tablet_manager.cc:1428] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 07:00:46.393177 3651 leader_election.cc:288] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 97909824241b492ea65d0366a55eb2f7 (127.4.40.1:38791), 31be710672c3434ba87eba80860734d4 (127.4.40.3:44699)
I20260811 07:00:46.403220 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.394076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:52353 (local address 127.4.40.1:38791)
0811 07:00:46.394334 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.394343 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.395372 (+ 1029us) server_negotiation.cc:407] Connection header received
0811 07:00:46.395696 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.395704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.395781 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.395891 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:46.396703 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.397900 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.400733 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.401970 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.402095 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.402213 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.402827 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.402958 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 07:00:46.403209 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.393529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:59639 (local address 127.4.40.3:44699)
0811 07:00:46.393751 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.393761 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.395638 (+ 1877us) server_negotiation.cc:407] Connection header received
0811 07:00:46.395901 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.395906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.395971 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.396067 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:46.396911 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.398063 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.400733 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.401961 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.402095 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.402213 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.402827 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.402958 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 07:00:46.403460 3655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.393634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44699 (local address 127.4.40.2:59639)
0811 07:00:46.395552 (+ 1918us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:46.395569 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:46.395756 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:46.396151 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:46.396156 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:46.396183 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:46.396731 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.396750 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.398192 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.398198 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.400575 (+ 2377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.400590 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.402269 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.402276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.402572 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.402697 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 07:00:46.402792 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:00:46.402987 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1780,"mutex_wait_us":201,"thread_start_us":1218,"threads_started":1}
I20260811 07:00:46.403494 3656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.393352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:38791 (local address 127.4.40.2:52353)
0811 07:00:46.395248 (+ 1896us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:46.395273 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:46.395525 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:46.396067 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:46.396072 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:46.396147 (+ 75us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:46.396558 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.396568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.398027 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.398031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.400576 (+ 2545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:46.400590 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.402272 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.402276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:46.402572 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.402693 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 07:00:46.402792 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 07:00:46.402983 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"mutex_wait_us":182,"thread_start_us":1871,"threads_started":1}
I20260811 07:00:46.404422 3517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f0d6bfb5b149cd87e83471fe05eccd" candidate_uuid: "62d6fd146a2b4a94907c1a9736180e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31be710672c3434ba87eba80860734d4" is_pre_election: true
I20260811 07:00:46.404434 2992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f0d6bfb5b149cd87e83471fe05eccd" candidate_uuid: "62d6fd146a2b4a94907c1a9736180e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97909824241b492ea65d0366a55eb2f7" is_pre_election: true
I20260811 07:00:46.404893 2992 raft_consensus.cc:2454] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62d6fd146a2b4a94907c1a9736180e6d in term 0.
I20260811 07:00:46.404896 3517 raft_consensus.cc:2454] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 62d6fd146a2b4a94907c1a9736180e6d in term 0.
I20260811 07:00:46.405500 3132 leader_election.cc:302] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31be710672c3434ba87eba80860734d4, 62d6fd146a2b4a94907c1a9736180e6d; no voters:
I20260811 07:00:46.405880 3651 raft_consensus.cc:2781] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:00:46.406033 3651 raft_consensus.cc:482] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:00:46.406144 3651 raft_consensus.cc:3037] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:46.408869 3651 raft_consensus.cc:504] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.409559 3651 leader_election.cc:288] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [CANDIDATE]: Term 1 election: Requested vote from peers 97909824241b492ea65d0366a55eb2f7 (127.4.40.1:38791), 31be710672c3434ba87eba80860734d4 (127.4.40.3:44699)
I20260811 07:00:46.409849 2992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f0d6bfb5b149cd87e83471fe05eccd" candidate_uuid: "62d6fd146a2b4a94907c1a9736180e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "97909824241b492ea65d0366a55eb2f7"
I20260811 07:00:46.410040 2992 raft_consensus.cc:3037] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:46.410029 3517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e9f0d6bfb5b149cd87e83471fe05eccd" candidate_uuid: "62d6fd146a2b4a94907c1a9736180e6d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31be710672c3434ba87eba80860734d4"
I20260811 07:00:46.410214 3517 raft_consensus.cc:3037] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:46.412272 2992 raft_consensus.cc:2454] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62d6fd146a2b4a94907c1a9736180e6d in term 1.
I20260811 07:00:46.412521 3517 raft_consensus.cc:2454] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 62d6fd146a2b4a94907c1a9736180e6d in term 1.
I20260811 07:00:46.412694 3127 leader_election.cc:302] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 62d6fd146a2b4a94907c1a9736180e6d, 97909824241b492ea65d0366a55eb2f7; no voters:
I20260811 07:00:46.412988 3651 raft_consensus.cc:2781] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:46.413399 3651 raft_consensus.cc:686] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 1 LEADER]: Becoming Leader. State: Replica: 62d6fd146a2b4a94907c1a9736180e6d, State: Running, Role: LEADER
I20260811 07:00:46.413861 3651 consensus_queue.cc:237] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } }
I20260811 07:00:46.418345 2744 catalog_manager.cc:5495] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d reported cstate change: term changed from 0 to 1, leader changed from <none> to 62d6fd146a2b4a94907c1a9736180e6d (127.4.40.2). New cstate: current_term: 1 leader_uuid: "62d6fd146a2b4a94907c1a9736180e6d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "62d6fd146a2b4a94907c1a9736180e6d" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 34821 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 } health_report { overall_health: UNKNOWN } } }
I20260811 07:00:46.438938 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.433208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.4.40.2:34821)
0811 07:00:46.433416 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.433422 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.433529 (+ 107us) server_negotiation.cc:407] Connection header received
0811 07:00:46.433763 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.433767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.433849 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.433995 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:00:46.434417 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.435238 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.436871 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.437282 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.437430 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.437923 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:00:46.438379 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:00:46.438544 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.438603 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.438678 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 07:00:46.445513 3347 tablet.cc:2369] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:46.446830 3517 raft_consensus.cc:1261] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 1 FOLLOWER]: Refusing update from remote peer 62d6fd146a2b4a94907c1a9736180e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:46.446873 2992 raft_consensus.cc:1261] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 1 FOLLOWER]: Refusing update from remote peer 62d6fd146a2b4a94907c1a9736180e6d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:46.447885 3657 consensus_queue.cc:1035] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "31be710672c3434ba87eba80860734d4" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:46.448251 3651 consensus_queue.cc:1035] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [LEADER]: Connected to new peer: Peer: permanent_uuid: "97909824241b492ea65d0366a55eb2f7" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 38791 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:46.458823 3661 mvcc.cc:205] Tried to move back new op lower bound from 7317224023836598272 to 7317224023714742272. Current Snapshot: MvccSnapshot[applied={T|T < 7317224023836598272}]
I20260811 07:00:46.472996 2748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:46.473222 2748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:46.476675 2748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 84780e205f534829ac99dbc92121310f: Sending DeleteTablet for 3 replicas of tablet e9f0d6bfb5b149cd87e83471fe05eccd
I20260811 07:00:46.477785 2961 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0d6bfb5b149cd87e83471fe05eccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:46 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:37422
I20260811 07:00:46.477947 3216 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0d6bfb5b149cd87e83471fe05eccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:46 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:51568
I20260811 07:00:46.478155 3485 tablet_service.cc:1514] Processing DeleteTablet for tablet e9f0d6bfb5b149cd87e83471fe05eccd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:46 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:54150
I20260811 07:00:46.480321 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 2847
I20260811 07:00:46.488807 3684 tablet_replica.cc:330] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: stopping tablet replica
I20260811 07:00:46.489408 3684 raft_consensus.cc:2229] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:00:46.490376 3684 raft_consensus.cc:2258] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:46.491981 3686 tablet_replica.cc:330] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: stopping tablet replica
I20260811 07:00:46.492477 3686 raft_consensus.cc:2229] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:46.493096 3686 raft_consensus.cc:2258] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:46.496824 3679 tablet_replica.cc:330] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: stopping tablet replica
I20260811 07:00:46.497053 3684 ts_tablet_manager.cc:1905] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:46.498008 3686 ts_tablet_manager.cc:1905] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:46.504652 3679 raft_consensus.cc:2229] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:46.504941 3684 ts_tablet_manager.cc:1918] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:46.505126 3684 log.cc:1196] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/e9f0d6bfb5b149cd87e83471fe05eccd
I20260811 07:00:46.505160 3686 ts_tablet_manager.cc:1918] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:46.505200 3679 raft_consensus.cc:2258] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:46.505360 3686 log.cc:1196] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/e9f0d6bfb5b149cd87e83471fe05eccd
I20260811 07:00:46.505775 3684 ts_tablet_manager.cc:1939] T e9f0d6bfb5b149cd87e83471fe05eccd P 62d6fd146a2b4a94907c1a9736180e6d: Deleting consensus metadata
I20260811 07:00:46.505968 3686 ts_tablet_manager.cc:1939] T e9f0d6bfb5b149cd87e83471fe05eccd P 31be710672c3434ba87eba80860734d4: Deleting consensus metadata
I20260811 07:00:46.507659 2717 catalog_manager.cc:4841] TS 62d6fd146a2b4a94907c1a9736180e6d (127.4.40.2:34821): tablet e9f0d6bfb5b149cd87e83471fe05eccd (table test-table [id=f6f64e30c65b4f65a3c4adcc3430af2f]) successfully deleted
I20260811 07:00:46.507988 2717 catalog_manager.cc:4841] TS 31be710672c3434ba87eba80860734d4 (127.4.40.3:44699): tablet e9f0d6bfb5b149cd87e83471fe05eccd (table test-table [id=f6f64e30c65b4f65a3c4adcc3430af2f]) successfully deleted
I20260811 07:00:46.511538 3679 ts_tablet_manager.cc:1905] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 07:00:46 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 07:00:46.519327 3679 ts_tablet_manager.cc:1918] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:46.519527 3679 log.cc:1196] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/e9f0d6bfb5b149cd87e83471fe05eccd
I20260811 07:00:46.520143 3679 ts_tablet_manager.cc:1939] T e9f0d6bfb5b149cd87e83471fe05eccd P 97909824241b492ea65d0366a55eb2f7: Deleting consensus metadata
I20260811 07:00:46.522233 2716 catalog_manager.cc:4841] TS 97909824241b492ea65d0366a55eb2f7 (127.4.40.1:38791): tablet e9f0d6bfb5b149cd87e83471fe05eccd (table test-table [id=f6f64e30c65b4f65a3c4adcc3430af2f]) successfully deleted
I20260811 07:00:46.524425 3634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.482105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37426 (local address 127.4.40.1:38791)
0811 07:00:46.482279 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.482285 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.494638 (+ 12353us) server_negotiation.cc:407] Connection header received
0811 07:00:46.495117 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.495124 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.495236 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.495312 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:46.497448 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.498582 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.509613 (+ 11031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.510321 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.510468 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.516350 (+ 5882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:46.516365 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:46.516388 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:46.516442 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:46.520741 (+ 4299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.521896 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.521905 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.521907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.522077 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.522746 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.522751 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.522754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.523131 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:46.523483 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.523798 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.524026 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:00:46.525059 3076 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:46.617591 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 2847
I20260811 07:00:46.653662 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 3097
Aug 11 07:00:46 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 07:00:46.670532 3635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.654539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51588 (local address 127.4.40.2:34821)
0811 07:00:46.654748 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.654754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.654777 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:00:46.654847 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.654852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.654935 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.655072 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:46.656578 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.657781 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.659080 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.659759 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.659899 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.664499 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:46.664517 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:46.664537 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:46.664602 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:46.667661 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.668510 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.668518 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.668520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.668658 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.669076 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.669079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.669081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.669422 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:46.669697 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.670033 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.670189 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 07:00:46.671185 3339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:46.785050 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 3097
I20260811 07:00:46.812635 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 3356
Aug 11 07:00:46 dist-test-slave-7vjv krb5kdc[2658](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431643, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 07:00:46.828400 3641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:46.813394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54160 (local address 127.4.40.3:44699)
0811 07:00:46.813598 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:46.813605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:46.813662 (+ 57us) server_negotiation.cc:407] Connection header received
0811 07:00:46.813866 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:46.813872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:46.813944 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:46.814057 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:46.815484 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.816293 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.817718 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:46.818134 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:46.818281 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:46.822736 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:46.822749 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:46.822765 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:46.822802 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:46.825614 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.826246 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.826253 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.826255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.826402 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:46.826790 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:46.826796 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:46.826799 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:46.827162 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:46.827453 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:46.827786 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 07:00:46.828057 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:00:46.829063 3593 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:46.918882 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 3356
I20260811 07:00:46.947465 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 2684
I20260811 07:00:46.948690 2800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:47.057512 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 2684
2026-08-11T07:00:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4761 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T07:00:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:00:47Z Disabled control of system clock
I20260811 07:00:47.166651 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:33293
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:33293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:00:47.502048 3713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:47.502352 3713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:47.502529 3713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:47.509550 3713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:00:47.509690 3713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:47.509765 3713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:47.509862 3713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:00:47.509940 3713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:00:47.519230 3713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:33293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:33293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:47.520653 3713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:47.522460 3713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:47.534086 3722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:47.534062 3718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:47.534294 3725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:47.535084 3713 server_base.cc:1034] running on GCE node
I20260811 07:00:47.535908 3713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:47.537427 3713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:47.538654 3713 hybrid_clock.cc:648] HybridClock initialized: now 1786431647538613 us; error 50 us; skew 500 ppm
I20260811 07:00:47.538996 3713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:47.541356 3713 webserver.cc:460] Webserver started at http://127.4.40.62:38357/ using document root <none> and password file <none>
I20260811 07:00:47.542057 3713 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:47.542184 3713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:47.542515 3713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:47.545437 3713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "c56ab81318be46b681f9caf220eaad1a"
format_stamp: "Formatted at 2026-08-11 07:00:47 on dist-test-slave-7vjv"
server_key: "4c5070334d353f1cbf452041f8c28d96"
server_key_iv: "d2fdb84c92f1cf1fd3d3e7d839713f10"
server_key_version: "encryptionkey@0"
I20260811 07:00:47.546257 3713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "c56ab81318be46b681f9caf220eaad1a"
format_stamp: "Formatted at 2026-08-11 07:00:47 on dist-test-slave-7vjv"
server_key: "4c5070334d353f1cbf452041f8c28d96"
server_key_iv: "d2fdb84c92f1cf1fd3d3e7d839713f10"
server_key_version: "encryptionkey@0"
I20260811 07:00:47.551280 3713 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 07:00:47.554577 3733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:47.555866 3713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:00:47.556046 3713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "c56ab81318be46b681f9caf220eaad1a"
format_stamp: "Formatted at 2026-08-11 07:00:47 on dist-test-slave-7vjv"
server_key: "4c5070334d353f1cbf452041f8c28d96"
server_key_iv: "d2fdb84c92f1cf1fd3d3e7d839713f10"
server_key_version: "encryptionkey@0"
I20260811 07:00:47.556319 3713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:47.574595 3713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:47.575358 3713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:47.575656 3713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:47.593894 3713 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:33293
I20260811 07:00:47.593919 3830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:33293 every 8 connection(s)
I20260811 07:00:47.595191 3713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:00:47.598824 3834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:47.602757 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 3713
I20260811 07:00:47.602949 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:00:47.603523 4256 external_mini_cluster.cc:1428] Setting key 667a5a19671f1536956f0a6bd2e8a7bc
I20260811 07:00:47.611770 3834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a: Bootstrap starting.
I20260811 07:00:47.615357 3834 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:47.616526 3834 log.cc:824] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:47.616866 3838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:47.605458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60014 (local address 127.4.40.62:33293)
0811 07:00:47.606412 (+ 954us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:47.606422 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:47.606468 (+ 46us) server_negotiation.cc:407] Connection header received
0811 07:00:47.607979 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:47.607999 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:47.608497 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:47.608802 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:47.609407 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:47.610646 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:47.612562 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:47.613203 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:47.613336 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:47.614601 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:47.614608 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:47.614620 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:47.615099 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:47.615202 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:47.615222 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:47.615309 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:47.615587 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 07:00:47.615909 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":398,"threads_started":1}
I20260811 07:00:47.619138 3834 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a: No bootstrap required, opened a new log
I20260811 07:00:47.626045 3834 raft_consensus.cc:348] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ab81318be46b681f9caf220eaad1a" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33293 } }
I20260811 07:00:47.626399 3834 raft_consensus.cc:374] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:47.626516 3834 raft_consensus.cc:729] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c56ab81318be46b681f9caf220eaad1a, State: Initialized, Role: FOLLOWER
I20260811 07:00:47.627008 3834 consensus_queue.cc:260] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ab81318be46b681f9caf220eaad1a" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33293 } }
I20260811 07:00:47.627264 3834 raft_consensus.cc:388] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:00:47.627401 3834 raft_consensus.cc:482] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:00:47.627562 3834 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:47.629664 3834 raft_consensus.cc:504] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ab81318be46b681f9caf220eaad1a" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33293 } }
I20260811 07:00:47.630131 3834 leader_election.cc:302] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c56ab81318be46b681f9caf220eaad1a; no voters:
I20260811 07:00:47.630712 3834 leader_election.cc:288] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:00:47.630841 3841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:47.631546 3841 raft_consensus.cc:686] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [term 1 LEADER]: Becoming Leader. State: Replica: c56ab81318be46b681f9caf220eaad1a, State: Running, Role: LEADER
I20260811 07:00:47.632041 3841 consensus_queue.cc:237] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ab81318be46b681f9caf220eaad1a" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33293 } }
I20260811 07:00:47.632722 3834 sys_catalog.cc:564] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:00:47.634742 3842 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c56ab81318be46b681f9caf220eaad1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ab81318be46b681f9caf220eaad1a" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33293 } } }
I20260811 07:00:47.635217 3842 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:47.635093 3845 sys_catalog.cc:455] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c56ab81318be46b681f9caf220eaad1a. Latest consensus state: current_term: 1 leader_uuid: "c56ab81318be46b681f9caf220eaad1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c56ab81318be46b681f9caf220eaad1a" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33293 } } }
I20260811 07:00:47.635417 3845 sys_catalog.cc:458] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:47.636370 3857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:00:47.641537 3857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:00:47.649466 3857 catalog_manager.cc:1347] Generated new cluster ID: b54eb0e794264133aa64349ec4b7e523
I20260811 07:00:47.649575 3857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:00:47.659191 3857 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:00:47.660959 3857 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:00:47.669147 3857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a: Generated new TSK 0
I20260811 07:00:47.670022 3857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:00:47.678974 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33293
--builtin_ntp_servers=127.4.40.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:00:48.026258 3874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:48.026543 3874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:48.026630 3874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:48.033703 3874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:48.033826 3874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:48.033931 3874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:00:48.043632 3874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:48.044893 3874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:48.046622 3874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:48.059717 3882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:48.059762 3883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:48.059959 3885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:48.060883 3874 server_base.cc:1034] running on GCE node
I20260811 07:00:48.061522 3874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:48.062405 3874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:48.063634 3874 hybrid_clock.cc:648] HybridClock initialized: now 1786431648063621 us; error 80 us; skew 500 ppm
I20260811 07:00:48.063962 3874 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:48.066355 3874 webserver.cc:460] Webserver started at http://127.4.40.1:45395/ using document root <none> and password file <none>
I20260811 07:00:48.067018 3874 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:48.067162 3874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:48.067494 3874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:48.070362 3874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "7144a03407794c12aea5be7ab677ee0d"
format_stamp: "Formatted at 2026-08-11 07:00:48 on dist-test-slave-7vjv"
server_key: "55dfa386fff9f180da48a0ebbbf692dc"
server_key_iv: "501033ef5fca245a3da405a8567e10f2"
server_key_version: "encryptionkey@0"
I20260811 07:00:48.071169 3874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "7144a03407794c12aea5be7ab677ee0d"
format_stamp: "Formatted at 2026-08-11 07:00:48 on dist-test-slave-7vjv"
server_key: "55dfa386fff9f180da48a0ebbbf692dc"
server_key_iv: "501033ef5fca245a3da405a8567e10f2"
server_key_version: "encryptionkey@0"
I20260811 07:00:48.076193 3874 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 07:00:48.079527 3898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:48.080861 3874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 07:00:48.081027 3874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "7144a03407794c12aea5be7ab677ee0d"
format_stamp: "Formatted at 2026-08-11 07:00:48 on dist-test-slave-7vjv"
server_key: "55dfa386fff9f180da48a0ebbbf692dc"
server_key_iv: "501033ef5fca245a3da405a8567e10f2"
server_key_version: "encryptionkey@0"
I20260811 07:00:48.081229 3874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:48.106848 3874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:48.107599 3874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:48.107822 3874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:48.108788 3874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:48.110208 3874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:48.110297 3874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:48.110378 3874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:48.110450 3874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:48.146271 3874 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:44807
I20260811 07:00:48.146350 4113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:44807 every 8 connection(s)
I20260811 07:00:48.147614 3874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:00:48.153013 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 3874
I20260811 07:00:48.153211 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 07:00:48.153649 4256 external_mini_cluster.cc:1428] Setting key 7ff589acd5d3dbaaf0628ac191dcb8f6
I20260811 07:00:48.156657 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33293
--builtin_ntp_servers=127.4.40.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:00:48.160342 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:48.150235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:38469 (local address 127.4.40.62:33293)
0811 07:00:48.150694 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:48.150698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:48.151759 (+ 1061us) server_negotiation.cc:407] Connection header received
0811 07:00:48.153741 (+ 1982us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:48.153747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:48.153837 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:48.153955 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:48.155111 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.155911 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.157570 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.157966 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.158102 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:48.159432 (+ 1330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:48.159447 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:48.159457 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:48.159475 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:48.159516 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:48.159528 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:48.159630 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:48.159855 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 07:00:48.159950 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":262,"threads_started":1}
I20260811 07:00:48.161312 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:48.150716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33293 (local address 127.4.40.1:38469)
0811 07:00:48.151607 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:48.151643 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:48.153470 (+ 1827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:48.154185 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:48.154193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:48.154276 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:48.154945 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:48.154963 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.156083 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.156087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:48.157430 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:48.157445 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.158196 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.158205 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:48.158428 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:48.159118 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:00:48.159120 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:48.159164 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:00:48.159238 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:48.159652 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:48.159668 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:48.159674 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:00:48.159842 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 07:00:48.160246 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":288,"threads_started":1}
I20260811 07:00:48.163167 4115 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33293
I20260811 07:00:48.163466 4115 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:48.164242 4115 heartbeater.cc:511] Master 127.4.40.62:33293 requested a full tablet report, sending...
I20260811 07:00:48.166761 3760 ts_manager.cc:194] Registered new tserver with Master: 7144a03407794c12aea5be7ab677ee0d (127.4.40.1:44807)
I20260811 07:00:48.169134 3760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.1:38469
W20260811 07:00:48.508786 4119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:48.509053 4119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:48.509227 4119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:48.516443 4119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:48.516544 4119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:48.516728 4119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:00:48.525894 4119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:48.527202 4119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:48.529093 4119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:48.542891 4129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:48.542943 4134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:48.542860 4131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:48.543649 4119 server_base.cc:1034] running on GCE node
I20260811 07:00:48.544381 4119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:48.545287 4119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:48.546494 4119 hybrid_clock.cc:648] HybridClock initialized: now 1786431648546451 us; error 35 us; skew 500 ppm
I20260811 07:00:48.546809 4119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:48.549108 4119 webserver.cc:460] Webserver started at http://127.4.40.2:38305/ using document root <none> and password file <none>
I20260811 07:00:48.549877 4119 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:48.550011 4119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:48.550365 4119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:48.553359 4119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "5796f9b6efdd4d2a8267fc0880238b5c"
format_stamp: "Formatted at 2026-08-11 07:00:48 on dist-test-slave-7vjv"
server_key: "ac37888e9893beaf2742bae77de9fcb7"
server_key_iv: "56cf967a3eb87299dd8024c966b230a0"
server_key_version: "encryptionkey@0"
I20260811 07:00:48.554198 4119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "5796f9b6efdd4d2a8267fc0880238b5c"
format_stamp: "Formatted at 2026-08-11 07:00:48 on dist-test-slave-7vjv"
server_key: "ac37888e9893beaf2742bae77de9fcb7"
server_key_iv: "56cf967a3eb87299dd8024c966b230a0"
server_key_version: "encryptionkey@0"
I20260811 07:00:48.559404 4119 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 07:00:48.562883 4146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:48.564242 4119 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:48.564411 4119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "5796f9b6efdd4d2a8267fc0880238b5c"
format_stamp: "Formatted at 2026-08-11 07:00:48 on dist-test-slave-7vjv"
server_key: "ac37888e9893beaf2742bae77de9fcb7"
server_key_iv: "56cf967a3eb87299dd8024c966b230a0"
server_key_version: "encryptionkey@0"
I20260811 07:00:48.564620 4119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:48.612005 4119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:48.613054 4119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:48.613332 4119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:48.614538 4119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:48.616448 4119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:48.616559 4119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:48.616704 4119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:48.616791 4119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:48.650231 4119 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:43683
I20260811 07:00:48.650280 4364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:43683 every 8 connection(s)
I20260811 07:00:48.651551 4119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:00:48.653158 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4119
I20260811 07:00:48.653414 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 07:00:48.653925 4256 external_mini_cluster.cc:1428] Setting key 861da2a4b2b994850d6890cd57c3d69d
I20260811 07:00:48.657434 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33293
--builtin_ntp_servers=127.4.40.20:43689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:00:48.666400 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:48.654273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:51627 (local address 127.4.40.62:33293)
0811 07:00:48.654490 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:48.654496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:48.656135 (+ 1639us) server_negotiation.cc:407] Connection header received
0811 07:00:48.658847 (+ 2712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:48.658851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:48.658915 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:48.659001 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:48.660435 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.661181 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.663574 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.663988 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.664145 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:48.665700 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:48.665704 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:48.665707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:48.665725 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:48.665756 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:48.665767 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:48.665838 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:48.666100 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 07:00:48.666178 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 07:00:48.667919 4367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:48.654878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33293 (local address 127.4.40.2:51627)
0811 07:00:48.655959 (+ 1081us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:48.656008 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:48.658551 (+ 2543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:48.659236 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:48.659251 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:48.659383 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:48.660230 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:48.660258 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.661337 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.661345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:48.663415 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:48.663432 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:48.664243 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:48.664253 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:48.664577 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:48.665353 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:00:48.665358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:48.665417 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:00:48.665520 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:48.665870 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:48.665885 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:48.665895 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 07:00:48.666078 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 07:00:48.666487 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":324,"threads_started":1}
I20260811 07:00:48.669399 4365 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33293
I20260811 07:00:48.669720 4365 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:48.670652 4365 heartbeater.cc:511] Master 127.4.40.62:33293 requested a full tablet report, sending...
I20260811 07:00:48.672297 3760 ts_manager.cc:194] Registered new tserver with Master: 5796f9b6efdd4d2a8267fc0880238b5c (127.4.40.2:43683)
I20260811 07:00:48.673121 3760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.2:51627
W20260811 07:00:49.001053 4368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:49.001322 4368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:49.001415 4368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:49.008661 4368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:49.008761 4368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:49.008865 4368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:00:49.017568 4368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33293
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:49.018812 4368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:49.020581 4368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:49.033782 4374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:49.033833 4375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:49.033874 4377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:49.035022 4368 server_base.cc:1034] running on GCE node
I20260811 07:00:49.035686 4368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:49.036729 4368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:49.037955 4368 hybrid_clock.cc:648] HybridClock initialized: now 1786431649037889 us; error 58 us; skew 500 ppm
I20260811 07:00:49.038266 4368 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:49.040622 4368 webserver.cc:460] Webserver started at http://127.4.40.3:37643/ using document root <none> and password file <none>
I20260811 07:00:49.041289 4368 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:49.041406 4368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:49.041723 4368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:49.044615 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "a928f35afa434240bacb44d1918e7a10"
format_stamp: "Formatted at 2026-08-11 07:00:49 on dist-test-slave-7vjv"
server_key: "494d12f7f8557f23c47b804684405b78"
server_key_iv: "fb97fbb699492caa73a5ffaf7ff87501"
server_key_version: "encryptionkey@0"
I20260811 07:00:49.045388 4368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "a928f35afa434240bacb44d1918e7a10"
format_stamp: "Formatted at 2026-08-11 07:00:49 on dist-test-slave-7vjv"
server_key: "494d12f7f8557f23c47b804684405b78"
server_key_iv: "fb97fbb699492caa73a5ffaf7ff87501"
server_key_version: "encryptionkey@0"
I20260811 07:00:49.050344 4368 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 07:00:49.053614 4387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:49.054809 4368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:00:49.054996 4368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "a928f35afa434240bacb44d1918e7a10"
format_stamp: "Formatted at 2026-08-11 07:00:49 on dist-test-slave-7vjv"
server_key: "494d12f7f8557f23c47b804684405b78"
server_key_iv: "fb97fbb699492caa73a5ffaf7ff87501"
server_key_version: "encryptionkey@0"
I20260811 07:00:49.055230 4368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:49.075034 4368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:49.075769 4368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:49.076063 4368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:49.077013 4368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:49.078475 4368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:49.078598 4368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:49.078714 4368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:49.078799 4368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:49.111634 4368 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:34137
I20260811 07:00:49.111713 4624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:34137 every 8 connection(s)
I20260811 07:00:49.112915 4368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:00:49.113158 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4368
I20260811 07:00:49.113327 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 07:00:49.113962 4256 external_mini_cluster.cc:1428] Setting key 636738ddd27f5509ee51aa6cae6a7152
I20260811 07:00:49.125808 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.115745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:34269 (local address 127.4.40.62:33293)
0811 07:00:49.115905 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.115909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.117713 (+ 1804us) server_negotiation.cc:407] Connection header received
0811 07:00:49.119499 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.119505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.119603 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.119718 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:49.120798 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.121547 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.123058 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.123486 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.123610 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.124986 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:49.124991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:49.124994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:49.125019 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:49.125051 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:49.125062 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:49.125159 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.125383 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.125483 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 07:00:49.126773 4629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.116456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33293 (local address 127.4.40.3:34269)
0811 07:00:49.117535 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.117591 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:49.119235 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:49.119947 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:49.119954 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:49.120037 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:49.120637 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.120654 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.121714 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.121717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.122910 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.122922 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.123711 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.123719 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.123952 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.124677 (+ 725us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:00:49.124680 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:49.124715 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:00:49.124788 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:49.125159 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:49.125173 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:49.125183 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 07:00:49.125353 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 07:00:49.125729 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":360,"threads_started":1}
I20260811 07:00:49.128223 4625 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33293
I20260811 07:00:49.128535 4625 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:49.129518 4625 heartbeater.cc:511] Master 127.4.40.62:33293 requested a full tablet report, sending...
I20260811 07:00:49.130776 3760 ts_manager.cc:194] Registered new tserver with Master: a928f35afa434240bacb44d1918e7a10 (127.4.40.3:34137)
I20260811 07:00:49.131495 3760 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.3:34269
I20260811 07:00:49.141404 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:00:49.151473 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.145525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.4.40.62:33293)
0811 07:00:49.145699 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.145707 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.146155 (+ 448us) server_negotiation.cc:407] Connection header received
0811 07:00:49.146360 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.146366 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.146433 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.146547 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:49.147017 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.147692 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.149151 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.149572 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.149711 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.150790 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:49.150795 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:49.150798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:49.150816 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:49.150845 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:49.150856 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:49.150946 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.151098 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.151194 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:00:49.154960 3760 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:00:49.162021 3760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:00:49.172151 4115 heartbeater.cc:503] Master 127.4.40.62:33293 was elected leader, sending a full tablet report...
I20260811 07:00:49.188283 4643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.174758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43318 (local address 127.4.40.2:43683)
0811 07:00:49.175498 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.175507 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.176221 (+ 714us) server_negotiation.cc:407] Connection header received
0811 07:00:49.176657 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.176667 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.176934 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.177095 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:49.178302 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.180427 (+ 2125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.184070 (+ 3643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.186249 (+ 2179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.186402 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.186700 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.187581 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.187813 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":418,"threads_started":1}
I20260811 07:00:49.189571 4644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.174514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:43683 (local address 127.0.0.1:43318)
0811 07:00:49.176053 (+ 1539us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.176122 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:49.176370 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:49.177457 (+ 1087us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:49.177465 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:49.177509 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:49.178030 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.178047 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.180666 (+ 2619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.180674 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.183699 (+ 3025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.183720 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.186929 (+ 3209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.186938 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.187246 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.187321 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 07:00:49.188861 (+ 1540us) client_negotiation.cc:240] Negotiation successful
0811 07:00:49.189153 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1270,"thread_start_us":2033,"threads_started":1}
I20260811 07:00:49.194371 4226 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcdaddfdcaf4302901e5b7ab3f01409 (DEFAULT_TABLE table=test-table [id=f7e876abb54a44139174995ae9ccf67e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:49.196009 4226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcdaddfdcaf4302901e5b7ab3f01409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:49.197919 4653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.177355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:34137 (local address 127.0.0.1:35104)
0811 07:00:49.189407 (+ 12052us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.189430 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:49.189575 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:49.191100 (+ 1525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:49.191105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:49.191131 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:49.191563 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.191578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.193733 (+ 2155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.193738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.195633 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.195662 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.197092 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.197098 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.197302 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.197335 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 07:00:49.197438 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 07:00:49.197558 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11875,"mutex_wait_us":27,"thread_start_us":12314,"threads_started":1}
I20260811 07:00:49.198660 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.177388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35104 (local address 127.4.40.3:34137)
0811 07:00:49.190632 (+ 13244us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.190647 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.190675 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:00:49.190748 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.190753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.190925 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.191116 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:49.191767 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.193569 (+ 1802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.195838 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.196893 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.197051 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.197318 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.198147 (+ 829us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.198305 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13082,"thread_start_us":12947,"threads_started":1}
I20260811 07:00:49.199570 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.175181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:46422)
0811 07:00:49.176825 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.176842 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:49.176947 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:49.192421 (+ 15474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:49.192426 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:49.192448 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:49.192815 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.192830 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.194734 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.194740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.197234 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.197250 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.199043 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.199048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.199158 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.199180 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:00:49.199228 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 07:00:49.199324 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1512,"mutex_wait_us":58,"thread_start_us":671,"threads_started":1}
I20260811 07:00:49.200568 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.175954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46422 (local address 127.4.40.1:44807)
0811 07:00:49.191810 (+ 15856us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.191816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.191862 (+ 46us) server_negotiation.cc:407] Connection header received
0811 07:00:49.191960 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.191966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.192217 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.193041 (+ 824us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:49.193231 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.194562 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.197440 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.198820 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.199838 (+ 1018us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.200088 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.200191 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.200312 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15691,"thread_start_us":18725,"threads_started":1}
I20260811 07:00:49.202673 4486 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcdaddfdcaf4302901e5b7ab3f01409 (DEFAULT_TABLE table=test-table [id=f7e876abb54a44139174995ae9ccf67e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:49.203186 3988 tablet_service.cc:1467] Processing CreateTablet for tablet 6dcdaddfdcaf4302901e5b7ab3f01409 (DEFAULT_TABLE table=test-table [id=f7e876abb54a44139174995ae9ccf67e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:49.204391 4486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcdaddfdcaf4302901e5b7ab3f01409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:49.204409 3988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dcdaddfdcaf4302901e5b7ab3f01409. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:49.213204 4655 tablet_bootstrap.cc:492] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Bootstrap starting.
I20260811 07:00:49.217010 4656 tablet_bootstrap.cc:492] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: Bootstrap starting.
I20260811 07:00:49.217645 4655 tablet_bootstrap.cc:654] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:49.218652 4655 log.cc:824] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:49.220876 4656 tablet_bootstrap.cc:654] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:49.221393 4655 tablet_bootstrap.cc:492] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: No bootstrap required, opened a new log
I20260811 07:00:49.221659 4655 ts_tablet_manager.cc:1397] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 07:00:49.222257 4656 log.cc:824] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:49.222628 4659 tablet_bootstrap.cc:492] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Bootstrap starting.
I20260811 07:00:49.226240 4659 tablet_bootstrap.cc:654] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:49.227084 4656 tablet_bootstrap.cc:492] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: No bootstrap required, opened a new log
I20260811 07:00:49.227483 4656 ts_tablet_manager.cc:1397] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 07:00:49.227521 4659 log.cc:824] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:49.230795 4659 tablet_bootstrap.cc:492] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: No bootstrap required, opened a new log
I20260811 07:00:49.231061 4659 ts_tablet_manager.cc:1397] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260811 07:00:49.232206 4655 raft_consensus.cc:348] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.232720 4655 raft_consensus.cc:374] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:49.232852 4655 raft_consensus.cc:729] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5796f9b6efdd4d2a8267fc0880238b5c, State: Initialized, Role: FOLLOWER
I20260811 07:00:49.233479 4655 consensus_queue.cc:260] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.234663 4365 heartbeater.cc:503] Master 127.4.40.62:33293 was elected leader, sending a full tablet report...
I20260811 07:00:49.235272 4655 ts_tablet_manager.cc:1428] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 07:00:49.238629 4656 raft_consensus.cc:348] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.239035 4656 raft_consensus.cc:374] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:49.239144 4656 raft_consensus.cc:729] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7144a03407794c12aea5be7ab677ee0d, State: Initialized, Role: FOLLOWER
I20260811 07:00:49.239781 4656 consensus_queue.cc:260] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.241683 4656 ts_tablet_manager.cc:1428] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 07:00:49.241678 4659 raft_consensus.cc:348] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.242118 4659 raft_consensus.cc:374] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:49.242274 4659 raft_consensus.cc:729] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a928f35afa434240bacb44d1918e7a10, State: Initialized, Role: FOLLOWER
I20260811 07:00:49.242911 4659 consensus_queue.cc:260] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.244136 4625 heartbeater.cc:503] Master 127.4.40.62:33293 was elected leader, sending a full tablet report...
I20260811 07:00:49.244650 4659 ts_tablet_manager.cc:1428] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Time spent starting tablet: real 0.013s user 0.015s sys 0.000s
I20260811 07:00:49.259155 4667 raft_consensus.cc:482] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:49.259375 4667 raft_consensus.cc:504] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.261211 4667 leader_election.cc:288] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5796f9b6efdd4d2a8267fc0880238b5c (127.4.40.2:43683), a928f35afa434240bacb44d1918e7a10 (127.4.40.3:34137)
I20260811 07:00:49.268868 4643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.261602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57745 (local address 127.4.40.2:43683)
0811 07:00:49.261841 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.261856 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.262108 (+ 252us) server_negotiation.cc:407] Connection header received
0811 07:00:49.262332 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.262341 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.262453 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.262572 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:49.263285 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.264283 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.266832 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.268000 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.268152 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.268262 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.268527 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.268643 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:00:49.269107 4672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.261425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:43683 (local address 127.4.40.1:57745)
0811 07:00:49.261983 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.261999 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:49.262162 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:49.262644 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:49.262650 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:49.262686 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:49.263132 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.263144 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.264401 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.264405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.266653 (+ 2248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.266668 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.268217 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.268222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.268348 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.268371 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:00:49.268487 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 07:00:49.268621 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":285,"threads_started":1}
I20260811 07:00:49.269223 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.262354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57793 (local address 127.4.40.3:34137)
0811 07:00:49.262589 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.262595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.262949 (+ 354us) server_negotiation.cc:407] Connection header received
0811 07:00:49.263113 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.263116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.263167 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.263287 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:49.263749 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.264667 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.267035 (+ 2368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.268322 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.268422 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.268535 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.268837 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.268983 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 07:00:49.269479 4673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.262149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:34137 (local address 127.4.40.1:57793)
0811 07:00:49.262854 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.262870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:49.262994 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:49.263323 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:49.263326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:49.263345 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:49.263609 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.263616 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.264801 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.264805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.266880 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:49.266890 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.268514 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.268521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:49.268678 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.268708 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 07:00:49.268793 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 07:00:49.269064 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":459,"threads_started":1}
I20260811 07:00:49.269949 4273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcdaddfdcaf4302901e5b7ab3f01409" candidate_uuid: "7144a03407794c12aea5be7ab677ee0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" is_pre_election: true
I20260811 07:00:49.270393 4273 raft_consensus.cc:2454] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7144a03407794c12aea5be7ab677ee0d in term 0.
I20260811 07:00:49.270419 4518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcdaddfdcaf4302901e5b7ab3f01409" candidate_uuid: "7144a03407794c12aea5be7ab677ee0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a928f35afa434240bacb44d1918e7a10" is_pre_election: true
I20260811 07:00:49.270900 4518 raft_consensus.cc:2454] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7144a03407794c12aea5be7ab677ee0d in term 0.
I20260811 07:00:49.271049 3904 leader_election.cc:302] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5796f9b6efdd4d2a8267fc0880238b5c, 7144a03407794c12aea5be7ab677ee0d; no voters:
I20260811 07:00:49.271412 4667 raft_consensus.cc:2781] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:00:49.271559 4667 raft_consensus.cc:482] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:00:49.271679 4667 raft_consensus.cc:3037] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:49.274046 4667 raft_consensus.cc:504] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.274616 4667 leader_election.cc:288] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [CANDIDATE]: Term 1 election: Requested vote from peers 5796f9b6efdd4d2a8267fc0880238b5c (127.4.40.2:43683), a928f35afa434240bacb44d1918e7a10 (127.4.40.3:34137)
I20260811 07:00:49.275058 4273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcdaddfdcaf4302901e5b7ab3f01409" candidate_uuid: "7144a03407794c12aea5be7ab677ee0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5796f9b6efdd4d2a8267fc0880238b5c"
I20260811 07:00:49.275128 4518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dcdaddfdcaf4302901e5b7ab3f01409" candidate_uuid: "7144a03407794c12aea5be7ab677ee0d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a928f35afa434240bacb44d1918e7a10"
I20260811 07:00:49.275264 4273 raft_consensus.cc:3037] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:49.275341 4518 raft_consensus.cc:3037] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:49.277611 4273 raft_consensus.cc:2454] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7144a03407794c12aea5be7ab677ee0d in term 1.
I20260811 07:00:49.277675 4518 raft_consensus.cc:2454] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7144a03407794c12aea5be7ab677ee0d in term 1.
I20260811 07:00:49.278090 3904 leader_election.cc:302] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5796f9b6efdd4d2a8267fc0880238b5c, 7144a03407794c12aea5be7ab677ee0d; no voters:
I20260811 07:00:49.278374 4667 raft_consensus.cc:2781] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:49.278796 4667 raft_consensus.cc:686] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 1 LEADER]: Becoming Leader. State: Replica: 7144a03407794c12aea5be7ab677ee0d, State: Running, Role: LEADER
I20260811 07:00:49.279237 4667 consensus_queue.cc:237] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } }
I20260811 07:00:49.283447 3760 catalog_manager.cc:5495] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7144a03407794c12aea5be7ab677ee0d (127.4.40.1). New cstate: current_term: 1 leader_uuid: "7144a03407794c12aea5be7ab677ee0d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7144a03407794c12aea5be7ab677ee0d" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 44807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 } health_report { overall_health: UNKNOWN } } }
I20260811 07:00:49.309092 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.303989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46428 (local address 127.4.40.1:44807)
0811 07:00:49.304198 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.304202 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.304219 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:00:49.304303 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.304307 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.304397 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.304539 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:00:49.304855 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.305532 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.307031 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.307436 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.307545 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.307981 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:00:49.308557 (+ 576us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:00:49.308678 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.308835 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.308899 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 07:00:49.317368 4518 raft_consensus.cc:1261] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 1 FOLLOWER]: Refusing update from remote peer 7144a03407794c12aea5be7ab677ee0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:49.317365 4273 raft_consensus.cc:1261] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 1 FOLLOWER]: Refusing update from remote peer 7144a03407794c12aea5be7ab677ee0d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:49.318324 4676 consensus_queue.cc:1035] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5796f9b6efdd4d2a8267fc0880238b5c" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 43683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:49.318670 4667 consensus_queue.cc:1035] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a928f35afa434240bacb44d1918e7a10" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 34137 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:49.330541 4683 mvcc.cc:205] Tried to move back new op lower bound from 7317224035594235904 to 7317224035451478016. Current Snapshot: MvccSnapshot[applied={T|T < 7317224035594235904}]
I20260811 07:00:49.330827 4686 mvcc.cc:205] Tried to move back new op lower bound from 7317224035594235904 to 7317224035451478016. Current Snapshot: MvccSnapshot[applied={T|T < 7317224035594235904}]
I20260811 07:00:49.345383 3760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:49.345625 3760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:49.348909 3760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c56ab81318be46b681f9caf220eaad1a: Sending DeleteTablet for 3 replicas of tablet 6dcdaddfdcaf4302901e5b7ab3f01409
I20260811 07:00:49.350092 3988 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dcdaddfdcaf4302901e5b7ab3f01409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:49 UTC) from {username='slave'} at 127.0.0.1:46422
I20260811 07:00:49.350102 4226 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dcdaddfdcaf4302901e5b7ab3f01409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:49 UTC) from {username='slave'} at 127.0.0.1:43318
I20260811 07:00:49.350497 4486 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dcdaddfdcaf4302901e5b7ab3f01409 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:49 UTC) from {username='slave'} at 127.0.0.1:35104
I20260811 07:00:49.352269 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 3874
I20260811 07:00:49.381626 4700 tablet_replica.cc:330] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: stopping tablet replica
I20260811 07:00:49.381772 4704 tablet_replica.cc:330] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: stopping tablet replica
I20260811 07:00:49.382285 4704 raft_consensus.cc:2229] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:49.382508 4700 raft_consensus.cc:2229] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:00:49.382862 4704 raft_consensus.cc:2258] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:49.383657 4700 raft_consensus.cc:2258] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:49.383703 4701 tablet_replica.cc:330] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: stopping tablet replica
I20260811 07:00:49.384140 4701 raft_consensus.cc:2229] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:49.384824 4701 raft_consensus.cc:2258] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:49.385517 4704 ts_tablet_manager.cc:1905] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:49.387192 4700 ts_tablet_manager.cc:1905] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 7144a03407794c12aea5be7ab677ee0d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:49.389146 4651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.353312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46432 (local address 127.4.40.1:44807)
0811 07:00:49.353501 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.353506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.374076 (+ 20570us) server_negotiation.cc:407] Connection header received
0811 07:00:49.374677 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.374685 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.374820 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.374920 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:49.375903 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.377546 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.384882 (+ 7336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.385566 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.385696 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.387301 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:49.387312 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:49.387329 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:49.388160 (+ 831us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:49.388233 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:49.388264 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:49.388355 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.388617 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.388724 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:00:49.390578 4100 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:49.392515 4701 ts_tablet_manager.cc:1905] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:49.393549 4704 ts_tablet_manager.cc:1918] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:49.393940 4704 log.cc:1196] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/6dcdaddfdcaf4302901e5b7ab3f01409
I20260811 07:00:49.394616 4704 ts_tablet_manager.cc:1939] T 6dcdaddfdcaf4302901e5b7ab3f01409 P a928f35afa434240bacb44d1918e7a10: Deleting consensus metadata
I20260811 07:00:49.397022 3737 catalog_manager.cc:4841] TS a928f35afa434240bacb44d1918e7a10 (127.4.40.3:34137): tablet 6dcdaddfdcaf4302901e5b7ab3f01409 (table test-table [id=f7e876abb54a44139174995ae9ccf67e]) successfully deleted
I20260811 07:00:49.399178 4701 ts_tablet_manager.cc:1918] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:49.399364 4701 log.cc:1196] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/6dcdaddfdcaf4302901e5b7ab3f01409
I20260811 07:00:49.399973 4701 ts_tablet_manager.cc:1939] T 6dcdaddfdcaf4302901e5b7ab3f01409 P 5796f9b6efdd4d2a8267fc0880238b5c: Deleting consensus metadata
I20260811 07:00:49.401643 3737 catalog_manager.cc:4841] TS 5796f9b6efdd4d2a8267fc0880238b5c (127.4.40.2:43683): tablet 6dcdaddfdcaf4302901e5b7ab3f01409 (table test-table [id=f7e876abb54a44139174995ae9ccf67e]) successfully deleted
I20260811 07:00:49.492376 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 3874
I20260811 07:00:49.520937 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4119
W20260811 07:00:49.520941 3738 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:44807 (error 108)
I20260811 07:00:49.522778 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.522350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43318)
0811 07:00:49.522515 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.522639 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 07:00:49.523012 3738 catalog_manager.cc:4584] TS 7144a03407794c12aea5be7ab677ee0d (127.4.40.1:44807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dcdaddfdcaf4302901e5b7ab3f01409: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
I20260811 07:00:49.528204 4643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.521718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54944 (local address 127.4.40.2:43683)
0811 07:00:49.521902 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.521908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.522048 (+ 140us) server_negotiation.cc:407] Connection header received
0811 07:00:49.522261 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.522266 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.522389 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.522497 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:49.522959 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.523753 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.525159 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.525562 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.525693 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.526714 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:49.526734 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:49.526747 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:49.527296 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:49.527344 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:49.527363 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:49.527481 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.527656 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.527760 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:00:49.528998 4359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:49.576893 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.576438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43320)
0811 07:00:49.576630 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.576718 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 07:00:49.577883 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.577560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43328)
0811 07:00:49.577701 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.577764 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 07:00:49.620416 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4119
I20260811 07:00:49.638285 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.637685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43342)
0811 07:00:49.637946 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.638070 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 07:00:49.639452 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.638975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43350)
0811 07:00:49.639155 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.639252 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:00:49.649430 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4368
I20260811 07:00:49.655982 4652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.650160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49176 (local address 127.4.40.3:34137)
0811 07:00:49.650335 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:49.650339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:49.650358 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:00:49.650502 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:49.650506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:49.650594 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:49.650690 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:49.651044 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.651778 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.653098 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:49.653492 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:49.653625 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:49.654670 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:49.654679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:49.654696 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:49.655283 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:49.655329 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:49.655348 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:49.655477 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:49.655597 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 07:00:49.655704 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 07:00:49.656723 4618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:49.720052 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.719504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43352)
0811 07:00:49.719748 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.719858 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 07:00:49.721179 4641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:49.720752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:44807 (local address 127.0.0.1:43364)
0811 07:00:49.720956 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:49.721043 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:44807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 07:00:49.746436 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4368
I20260811 07:00:49.774457 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 3713
I20260811 07:00:49.775568 3826 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:49.883687 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 3713
2026-08-11T07:00:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2820 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:00:50 dist-test-slave-7vjv krb5kdc[4724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:00:50 dist-test-slave-7vjv krb5kdc[4724](info): set up 2 sockets
Aug 11 07:00:50 dist-test-slave-7vjv krb5kdc[4724](info): commencing operation
krb5kdc: starting...
W20260811 07:00:51.290527 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.354s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:00:51 dist-test-slave-7vjv krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:00:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:00:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:00:51.611136 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45809
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45809
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:51.944960 4743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:51.945266 4743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:51.945441 4743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:51.952396 4743 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:00:51.952497 4743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:51.952551 4743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:51.952606 4743 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:00:51.952642 4743 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:00:51.961897 4743 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:51.962996 4743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:51.964248 4743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:51.975651 4753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:51.975564 4756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:51.975986 4764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:51.976536 4743 server_base.cc:1034] running on GCE node
I20260811 07:00:51.977358 4743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:51.978708 4743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:51.979931 4743 hybrid_clock.cc:648] HybridClock initialized: now 1786431651979887 us; error 52 us; skew 500 ppm
I20260811 07:00:51.980315 4743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:51 dist-test-slave-7vjv krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431651, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:51.985939 4743 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:00:51.987389 4743 webserver.cc:460] Webserver started at http://127.4.40.62:36775/ using document root <none> and password file <none>
I20260811 07:00:51.988056 4743 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:51.988237 4743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:51.988497 4743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:51.991647 4743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "0183910eec014136a1ba45da7902b905"
format_stamp: "Formatted at 2026-08-11 07:00:51 on dist-test-slave-7vjv"
server_key: "4406526fed1fbef3a66c027ecbb3d848"
server_key_iv: "99c2b8307b89b7558056df033d68b494"
server_key_version: "encryptionkey@0"
I20260811 07:00:51.992426 4743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "0183910eec014136a1ba45da7902b905"
format_stamp: "Formatted at 2026-08-11 07:00:51 on dist-test-slave-7vjv"
server_key: "4406526fed1fbef3a66c027ecbb3d848"
server_key_iv: "99c2b8307b89b7558056df033d68b494"
server_key_version: "encryptionkey@0"
I20260811 07:00:51.997342 4743 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 07:00:52.000562 4775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:52.001784 4743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:00:52.001960 4743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "0183910eec014136a1ba45da7902b905"
format_stamp: "Formatted at 2026-08-11 07:00:51 on dist-test-slave-7vjv"
server_key: "4406526fed1fbef3a66c027ecbb3d848"
server_key_iv: "99c2b8307b89b7558056df033d68b494"
server_key_version: "encryptionkey@0"
I20260811 07:00:52.002177 4743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:52.022224 4743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:52.026465 4743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:52.026762 4743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:52.044454 4743 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45809
I20260811 07:00:52.044499 4871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45809 every 8 connection(s)
I20260811 07:00:52.045887 4743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:00:52.049320 4872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:52.052227 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4743
I20260811 07:00:52.052448 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:00:52.052915 4256 external_mini_cluster.cc:1428] Setting key 6e2c7845c73594d98c462854e199f262
I20260811 07:00:52.061256 4872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905: Bootstrap starting.
I20260811 07:00:52.064951 4872 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:52.066256 4872 log.cc:824] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:00:52 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:52.069113 4872 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905: No bootstrap required, opened a new log
I20260811 07:00:52.077111 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:52.054744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.4.40.62:45809)
0811 07:00:52.055514 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:52.055525 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:52.055565 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:00:52.057210 (+ 1645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:52.057230 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:52.057626 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:52.057928 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:52.059774 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:52.061116 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:52.063042 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:52.063681 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:52.063805 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:52.068606 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:52.068617 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:52.068634 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:52.068679 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:52.072609 (+ 3930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:52.073335 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:52.073344 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:52.073352 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:52.073570 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:52.073930 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:52.073935 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:52.073938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:52.074639 (+ 701us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:52.074954 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:52.075247 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 07:00:52.075713 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":372,"threads_started":1}
I20260811 07:00:52.079823 4872 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0183910eec014136a1ba45da7902b905" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45809 } }
I20260811 07:00:52.080235 4872 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:52.080346 4872 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0183910eec014136a1ba45da7902b905, State: Initialized, Role: FOLLOWER
I20260811 07:00:52.080839 4872 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0183910eec014136a1ba45da7902b905" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45809 } }
I20260811 07:00:52.081068 4872 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:00:52.081195 4872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:00:52.081375 4872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:52.083670 4872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0183910eec014136a1ba45da7902b905" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45809 } }
I20260811 07:00:52.084185 4872 leader_election.cc:302] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0183910eec014136a1ba45da7902b905; no voters:
I20260811 07:00:52.084746 4872 leader_election.cc:288] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:00:52.084883 4880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:52.086236 4880 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [term 1 LEADER]: Becoming Leader. State: Replica: 0183910eec014136a1ba45da7902b905, State: Running, Role: LEADER
I20260811 07:00:52.086812 4872 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:00:52.086745 4880 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0183910eec014136a1ba45da7902b905" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45809 } }
I20260811 07:00:52.089408 4883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0183910eec014136a1ba45da7902b905. Latest consensus state: current_term: 1 leader_uuid: "0183910eec014136a1ba45da7902b905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0183910eec014136a1ba45da7902b905" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45809 } } }
I20260811 07:00:52.089754 4883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:52.090286 4882 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0183910eec014136a1ba45da7902b905" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0183910eec014136a1ba45da7902b905" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45809 } } }
I20260811 07:00:52.090649 4882 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905 [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:52.092959 4906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:00:52.097749 4906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:00:52.105970 4906 catalog_manager.cc:1347] Generated new cluster ID: 5e7bcf564e584fc3a1d72c499000cc82
I20260811 07:00:52.106102 4906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:00:52.115983 4906 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:00:52.117236 4906 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:00:52.127610 4906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905: Generated new TSK 0
I20260811 07:00:52.128386 4906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 07:00:52.213187 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45809
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:52.573575 4929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:52.573838 4929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:52.573963 4929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:52.581699 4929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:52.581813 4929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:52.582032 4929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:00:52.592733 4929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:52.594000 4929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:52.595396 4929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:52.609124 4937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:52.609157 4941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:52.609306 4939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:52.610435 4929 server_base.cc:1034] running on GCE node
I20260811 07:00:52.611069 4929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:52.612069 4929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:52.613336 4929 hybrid_clock.cc:648] HybridClock initialized: now 1786431652613289 us; error 44 us; skew 500 ppm
I20260811 07:00:52.613658 4929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:52 dist-test-slave-7vjv krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431652, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:52.616981 4929 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 07:00:52.618409 4929 webserver.cc:460] Webserver started at http://127.4.40.1:39431/ using document root <none> and password file <none>
I20260811 07:00:52.619145 4929 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:52.619284 4929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:52.619704 4929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:52.622879 4929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "0f421251d0c4490095d22a11a78d806a"
format_stamp: "Formatted at 2026-08-11 07:00:52 on dist-test-slave-7vjv"
server_key: "3f19aab0fb872d59cd726e199ed9528e"
server_key_iv: "02309e93f476e13b04f91c1f85f9d665"
server_key_version: "encryptionkey@0"
I20260811 07:00:52.623710 4929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "0f421251d0c4490095d22a11a78d806a"
format_stamp: "Formatted at 2026-08-11 07:00:52 on dist-test-slave-7vjv"
server_key: "3f19aab0fb872d59cd726e199ed9528e"
server_key_iv: "02309e93f476e13b04f91c1f85f9d665"
server_key_version: "encryptionkey@0"
I20260811 07:00:52.629942 4929 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 07:00:52.633735 4954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:52.635138 4929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 07:00:52.635439 4929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "0f421251d0c4490095d22a11a78d806a"
format_stamp: "Formatted at 2026-08-11 07:00:52 on dist-test-slave-7vjv"
server_key: "3f19aab0fb872d59cd726e199ed9528e"
server_key_iv: "02309e93f476e13b04f91c1f85f9d665"
server_key_version: "encryptionkey@0"
I20260811 07:00:52.635669 4929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:52.669955 4929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:52.674758 4929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:52.675089 4929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:52.676241 4929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:52.677742 4929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:52.677825 4929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:52.678017 4929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:52.678107 4929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:52.714879 4929 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:35785
I20260811 07:00:52.714924 5178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:35785 every 8 connection(s)
I20260811 07:00:52.716361 4929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:00:52.718899 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 4929
I20260811 07:00:52.719096 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 07:00:52.719700 4256 external_mini_cluster.cc:1428] Setting key 1533809ad1ad0773e7584433b4f378a4
Aug 11 07:00:52 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431652, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:52.737977 5183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:52.719024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:33721 (local address 127.4.40.62:45809)
0811 07:00:52.719547 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:52.719555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:52.720618 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 07:00:52.722891 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:52.722896 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:52.722988 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:52.723115 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:52.724660 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:52.725409 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:52.726922 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:52.727346 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:52.727482 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:52.731490 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:52.731495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:52.731498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:52.731535 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:52.734062 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:52.735085 (+ 1023us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:52.735089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:52.735091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:52.735257 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:52.735786 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:52.735792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:52.735797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:52.736143 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:52.736280 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:52.737285 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0811 07:00:52.737541 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":242,"threads_started":1}
I20260811 07:00:52.739023 5184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:52.719566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45809 (local address 127.4.40.1:33721)
0811 07:00:52.720446 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:52.720494 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:52.722639 (+ 2145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:52.723350 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:52.723358 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:52.723864 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:52.724493 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:52.724512 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:52.725569 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:52.725574 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:52.726754 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:52.726767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:52.727588 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:52.727598 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:52.727831 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:52.728552 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:52.728578 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:52.731272 (+ 2694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:52.734215 (+ 2943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:52.734245 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:52.734262 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:52.734958 (+ 696us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:52.735375 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:52.735383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:52.735389 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:52.735661 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:52.736280 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:52.736291 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:52.736755 (+ 464us) client_negotiation.cc:769] Sending connection context
0811 07:00:52.737145 (+ 390us) client_negotiation.cc:240] Negotiation successful
0811 07:00:52.737672 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":364,"threads_started":1}
I20260811 07:00:52.740667 5181 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45809
I20260811 07:00:52.741039 5181 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:52.741885 5181 heartbeater.cc:511] Master 127.4.40.62:45809 requested a full tablet report, sending...
I20260811 07:00:52.744223 4812 ts_manager.cc:194] Registered new tserver with Master: 0f421251d0c4490095d22a11a78d806a (127.4.40.1:35785)
I20260811 07:00:52.745666 4812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:33721
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 07:00:52.791262 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45809
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:53.137550 5187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:53.137840 5187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:53.138082 5187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:53.146076 5187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:53.146193 5187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:53.146538 5187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:00:53.158392 5187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:53.159780 5187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:53.161257 5187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:53.174759 5196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:53.174789 5200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:53.174813 5203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:53.176368 5187 server_base.cc:1034] running on GCE node
I20260811 07:00:53.177007 5187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:53.177983 5187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:53.179234 5187 hybrid_clock.cc:648] HybridClock initialized: now 1786431653179184 us; error 57 us; skew 500 ppm
I20260811 07:00:53.179556 5187 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:53 dist-test-slave-7vjv krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431653, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:53.182876 5187 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 07:00:53.184419 5187 webserver.cc:460] Webserver started at http://127.4.40.2:37567/ using document root <none> and password file <none>
I20260811 07:00:53.185133 5187 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:53.185349 5187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:53.185714 5187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:53.189381 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "864cced67bda44d989ad27b88f7ff3a0"
format_stamp: "Formatted at 2026-08-11 07:00:53 on dist-test-slave-7vjv"
server_key: "fba341035247c9688cc9d700ec1c930a"
server_key_iv: "dc07d5d817a182551105c799cdcd661e"
server_key_version: "encryptionkey@0"
I20260811 07:00:53.190236 5187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "864cced67bda44d989ad27b88f7ff3a0"
format_stamp: "Formatted at 2026-08-11 07:00:53 on dist-test-slave-7vjv"
server_key: "fba341035247c9688cc9d700ec1c930a"
server_key_iv: "dc07d5d817a182551105c799cdcd661e"
server_key_version: "encryptionkey@0"
I20260811 07:00:53.196488 5187 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 07:00:53.199846 5221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:53.201236 5187 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:53.201431 5187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "864cced67bda44d989ad27b88f7ff3a0"
format_stamp: "Formatted at 2026-08-11 07:00:53 on dist-test-slave-7vjv"
server_key: "fba341035247c9688cc9d700ec1c930a"
server_key_iv: "dc07d5d817a182551105c799cdcd661e"
server_key_version: "encryptionkey@0"
I20260811 07:00:53.201632 5187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:53.217952 5187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:53.221892 5187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:53.222128 5187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:53.223012 5187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:53.224314 5187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:53.224393 5187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:53.224481 5187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:53.224529 5187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:53.258800 5187 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:35109
I20260811 07:00:53.258841 5415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:35109 every 8 connection(s)
I20260811 07:00:53.260218 5187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:00:53.261508 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5187
I20260811 07:00:53.261708 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 07:00:53.262182 4256 external_mini_cluster.cc:1428] Setting key d1896b29786de342a6e3fd2ac636b920
Aug 11 07:00:53 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431653, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:53.281432 5421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.262882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:51979 (local address 127.4.40.62:45809)
0811 07:00:53.263377 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:53.263384 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:53.264555 (+ 1171us) server_negotiation.cc:407] Connection header received
0811 07:00:53.266354 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:53.266361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:53.266456 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:53.266609 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:53.268004 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.269142 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.270561 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.271227 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.271363 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.275309 (+ 3946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:53.275314 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:53.275317 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:53.275351 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:53.278111 (+ 2760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:53.279021 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:53.279025 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:53.279027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:53.279156 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:53.279607 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:53.279613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:53.279619 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:53.280000 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:53.280184 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:53.280850 (+ 666us) server_negotiation.cc:299] Negotiation successful
0811 07:00:53.281078 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":242,"threads_started":1}
I20260811 07:00:53.282389 5422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.263492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45809 (local address 127.4.40.2:51979)
0811 07:00:53.264380 (+ 888us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:53.264425 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:53.266073 (+ 1648us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:53.266789 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:53.266797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:53.267284 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:53.267824 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.267840 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.269295 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.269299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.270412 (+ 1113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.270425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.271433 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.271437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.271625 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.272339 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:53.272359 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:53.275066 (+ 2707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:53.278249 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:53.278260 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:53.278277 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:53.278893 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:53.279299 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:53.279302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:53.279304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:53.279481 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:53.280175 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:53.280182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:53.280471 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 07:00:53.280764 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 07:00:53.281240 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":345,"threads_started":1}
I20260811 07:00:53.283812 5416 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45809
I20260811 07:00:53.284191 5416 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:53.285254 5416 heartbeater.cc:511] Master 127.4.40.62:45809 requested a full tablet report, sending...
I20260811 07:00:53.286846 4812 ts_manager.cc:194] Registered new tserver with Master: 864cced67bda44d989ad27b88f7ff3a0 (127.4.40.2:35109)
I20260811 07:00:53.287613 4812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:51979
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 07:00:53.338677 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45809
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:35831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:00:53.677333 5428 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:53.677644 5428 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:53.677801 5428 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:53.684962 5428 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:53.685114 5428 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:53.685302 5428 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:00:53.695140 5428 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:35831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45809
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:53.696408 5428 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:53.697664 5428 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:53.710954 5438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:53.710891 5440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:53.710927 5437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:53.712057 5428 server_base.cc:1034] running on GCE node
I20260811 07:00:53.712709 5428 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:53.713660 5428 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:53.714922 5428 hybrid_clock.cc:648] HybridClock initialized: now 1786431653714861 us; error 59 us; skew 500 ppm
I20260811 07:00:53.715315 5428 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:00:53 dist-test-slave-7vjv krb5kdc[4724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431653, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:00:53.719285 5428 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 07:00:53.720831 5428 webserver.cc:460] Webserver started at http://127.4.40.3:44727/ using document root <none> and password file <none>
I20260811 07:00:53.721511 5428 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:53.721645 5428 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:53.721979 5428 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:53.724913 5428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "44ff4cd489ab4a41aee2090c2b58f84c"
format_stamp: "Formatted at 2026-08-11 07:00:53 on dist-test-slave-7vjv"
server_key: "507780d437cb54c63f7ee0f178780310"
server_key_iv: "4861a14e64ed012d8aa2aae314e0cd0a"
server_key_version: "encryptionkey@0"
I20260811 07:00:53.725692 5428 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "44ff4cd489ab4a41aee2090c2b58f84c"
format_stamp: "Formatted at 2026-08-11 07:00:53 on dist-test-slave-7vjv"
server_key: "507780d437cb54c63f7ee0f178780310"
server_key_iv: "4861a14e64ed012d8aa2aae314e0cd0a"
server_key_version: "encryptionkey@0"
I20260811 07:00:53.730784 5428 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 07:00:53.734179 5451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:53.735386 5428 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:00:53.735563 5428 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "44ff4cd489ab4a41aee2090c2b58f84c"
format_stamp: "Formatted at 2026-08-11 07:00:53 on dist-test-slave-7vjv"
server_key: "507780d437cb54c63f7ee0f178780310"
server_key_iv: "4861a14e64ed012d8aa2aae314e0cd0a"
server_key_version: "encryptionkey@0"
I20260811 07:00:53.735767 5428 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:53.749269 5181 heartbeater.cc:503] Master 127.4.40.62:45809 was elected leader, sending a full tablet report...
I20260811 07:00:53.760809 5428 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:53.765153 5428 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:53.765458 5428 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:53.766467 5428 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:53.767838 5428 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:53.767939 5428 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:53.768085 5428 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:53.768189 5428 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 07:00:53.801848 5428 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:35699
I20260811 07:00:53.801903 5652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:35699 every 8 connection(s)
I20260811 07:00:53.803120 5428 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:00:53.808502 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5428
I20260811 07:00:53.808673 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 07:00:53.809126 4256 external_mini_cluster.cc:1428] Setting key 7a5daafe1de17eec1554cadb5252293a
Aug 11 07:00:53 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431653, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:00:53.823691 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.805776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:59639 (local address 127.4.40.62:45809)
0811 07:00:53.806287 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:53.806292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:53.807139 (+ 847us) server_negotiation.cc:407] Connection header received
0811 07:00:53.809224 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:53.809229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:53.809335 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:53.809469 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:53.811593 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.812472 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.814012 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.814417 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.814563 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.818028 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:53.818033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:53.818037 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:53.818072 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:53.820825 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:53.821503 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:53.821507 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:53.821509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:53.821621 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:53.821999 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:53.822002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:53.822004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:53.822253 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:53.822364 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:53.823058 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 07:00:53.823316 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":284,"threads_started":1}
I20260811 07:00:53.824797 5661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.806315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45809 (local address 127.4.40.3:59639)
0811 07:00:53.806973 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:53.807028 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:53.808945 (+ 1917us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:53.809734 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:53.809742 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:53.810520 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:53.811419 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.811440 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.812628 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.812634 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.813852 (+ 1218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.813864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.814659 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.814669 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.814898 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.815518 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:00:53.815540 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:53.817824 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:53.820957 (+ 3133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:53.820963 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:53.820974 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:53.821400 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:53.821716 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:00:53.821719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:00:53.821721 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:00:53.821897 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:00:53.822364 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:53.822369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:53.822676 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 07:00:53.822936 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 07:00:53.823478 (+ 542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":284,"threads_started":1}
I20260811 07:00:53.826130 5655 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45809
I20260811 07:00:53.826421 5655 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:53.827093 5655 heartbeater.cc:511] Master 127.4.40.62:45809 requested a full tablet report, sending...
I20260811 07:00:53.828439 4812 ts_manager.cc:194] Registered new tserver with Master: 44ff4cd489ab4a41aee2090c2b58f84c (127.4.40.3:35699)
I20260811 07:00:53.829205 4812 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:59639
I20260811 07:00:53.836580 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:00:53.854543 5658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.841039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.4.40.62:45809)
0811 07:00:53.841202 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:53.841206 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:53.841690 (+ 484us) server_negotiation.cc:407] Connection header received
0811 07:00:53.841876 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:53.841884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:53.841948 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:53.842067 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:53.843550 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.844378 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.845668 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.846073 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.846211 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.849110 (+ 2899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:53.849115 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:53.849118 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:53.849148 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:53.851635 (+ 2487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:53.852435 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:53.852441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:53.852444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:53.852575 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:53.853016 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:53.853021 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:53.853024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:53.853321 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:53.853631 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:53.854067 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 07:00:53.854259 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:00:53.858778 4812 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:00:53.865525 4812 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:00:53.888820 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.877565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.4.40.3:35699)
0811 07:00:53.880121 (+ 2556us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:53.880132 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:53.880172 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:00:53.880280 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:53.880290 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:53.880559 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:53.881539 (+ 980us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:53.881829 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.883373 (+ 1544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.886240 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.887619 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.887744 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.887974 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:53.888298 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 07:00:53.888467 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2360,"thread_start_us":2199,"threads_started":1}
I20260811 07:00:53.889453 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.877919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44574 (local address 127.4.40.2:35109)
0811 07:00:53.879107 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:53.879118 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:53.879465 (+ 347us) server_negotiation.cc:407] Connection header received
0811 07:00:53.880761 (+ 1296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:53.880774 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:53.881035 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:53.881254 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:53.882299 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.883707 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.886490 (+ 2783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.887844 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.888677 (+ 833us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.888887 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:53.888975 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 07:00:53.889136 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1011,"thread_start_us":866,"threads_started":1}
I20260811 07:00:53.890219 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.877287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:35699 (local address 127.0.0.1:38238)
0811 07:00:53.878287 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:53.878336 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:53.878487 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:53.880806 (+ 2319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:53.880813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:53.880847 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:53.881324 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.881337 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.883528 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.883533 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.886088 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.886104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.887799 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.887804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.887980 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.888034 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 07:00:53.888233 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 07:00:53.888414 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"mutex_wait_us":89,"thread_start_us":795,"threads_started":1}
I20260811 07:00:53.890211 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.877613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:35109 (local address 127.0.0.1:44574)
0811 07:00:53.879336 (+ 1723us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:53.879350 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:53.880470 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:53.881546 (+ 1076us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:53.881554 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:53.881588 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:53.882085 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.882100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.883907 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.883916 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.886330 (+ 2414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.886350 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.888066 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.888071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.888313 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.888345 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 07:00:53.888409 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 07:00:53.888522 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1634,"mutex_wait_us":30,"spinlock_wait_cycles":11392,"thread_start_us":1897,"threads_started":1}
I20260811 07:00:53.890818 5686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.879889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:35785 (local address 127.0.0.1:43644)
0811 07:00:53.880478 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:53.880498 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:53.881806 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:53.883164 (+ 1358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:53.883169 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:53.883195 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:53.883630 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.883646 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.885638 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.885645 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.887399 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:53.887411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.889594 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.889600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:53.889778 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.889816 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:00:53.889913 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 07:00:53.890067 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":22,"spinlock_wait_cycles":1280,"thread_start_us":331,"threads_started":1}
I20260811 07:00:53.891425 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:53.880113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.4.40.1:35785)
0811 07:00:53.882754 (+ 2641us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:53.882760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:53.882780 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:00:53.882863 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:53.882867 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:53.882995 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:53.883181 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:53.883827 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.885473 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.887564 (+ 2091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:53.889101 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:53.889235 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:53.889458 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:53.889991 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 07:00:53.890184 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2506,"thread_start_us":2316,"threads_started":1}
I20260811 07:00:53.894244 5305 tablet_service.cc:1467] Processing CreateTablet for tablet 5560b20860b94035a867c5cecf6cb55f (DEFAULT_TABLE table=test-table [id=c217dc9c7476474bad165117ecb4701b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:53.894248 5545 tablet_service.cc:1467] Processing CreateTablet for tablet 5560b20860b94035a867c5cecf6cb55f (DEFAULT_TABLE table=test-table [id=c217dc9c7476474bad165117ecb4701b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:53.895788 5305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5560b20860b94035a867c5cecf6cb55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:53.895788 5545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5560b20860b94035a867c5cecf6cb55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:53.896833 5061 tablet_service.cc:1467] Processing CreateTablet for tablet 5560b20860b94035a867c5cecf6cb55f (DEFAULT_TABLE table=test-table [id=c217dc9c7476474bad165117ecb4701b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:53.898028 5061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5560b20860b94035a867c5cecf6cb55f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:53.910480 5689 tablet_bootstrap.cc:492] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Bootstrap starting.
I20260811 07:00:53.912925 5690 tablet_bootstrap.cc:492] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Bootstrap starting.
I20260811 07:00:53.913806 5692 tablet_bootstrap.cc:492] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Bootstrap starting.
I20260811 07:00:53.915092 5689 tablet_bootstrap.cc:654] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:53.916608 5689 log.cc:824] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:53.917819 5690 tablet_bootstrap.cc:654] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:53.919217 5692 tablet_bootstrap.cc:654] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:53.919199 5690 log.cc:824] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:53.920274 5689 tablet_bootstrap.cc:492] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: No bootstrap required, opened a new log
I20260811 07:00:53.920408 5692 log.cc:824] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:53.920605 5689 ts_tablet_manager.cc:1397] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.002s
I20260811 07:00:53.922662 5690 tablet_bootstrap.cc:492] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: No bootstrap required, opened a new log
I20260811 07:00:53.923020 5690 ts_tablet_manager.cc:1397] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260811 07:00:53.923177 5692 tablet_bootstrap.cc:492] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: No bootstrap required, opened a new log
I20260811 07:00:53.923437 5692 ts_tablet_manager.cc:1397] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 07:00:53.930648 5692 raft_consensus.cc:348] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:53.930987 5692 raft_consensus.cc:374] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:53.931118 5692 raft_consensus.cc:729] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44ff4cd489ab4a41aee2090c2b58f84c, State: Initialized, Role: FOLLOWER
I20260811 07:00:53.931707 5692 consensus_queue.cc:260] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:53.931751 5689 raft_consensus.cc:348] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:53.932266 5689 raft_consensus.cc:374] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:53.932550 5689 raft_consensus.cc:729] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f421251d0c4490095d22a11a78d806a, State: Initialized, Role: FOLLOWER
I20260811 07:00:53.932801 5655 heartbeater.cc:503] Master 127.4.40.62:45809 was elected leader, sending a full tablet report...
I20260811 07:00:53.933425 5692 ts_tablet_manager.cc:1428] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Time spent starting tablet: real 0.010s user 0.000s sys 0.010s
I20260811 07:00:53.933266 5689 consensus_queue.cc:260] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:53.934262 5690 raft_consensus.cc:348] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:53.934695 5690 raft_consensus.cc:374] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:53.934799 5690 raft_consensus.cc:729] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 864cced67bda44d989ad27b88f7ff3a0, State: Initialized, Role: FOLLOWER
I20260811 07:00:53.935048 5689 ts_tablet_manager.cc:1428] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Time spent starting tablet: real 0.014s user 0.008s sys 0.006s
I20260811 07:00:53.935494 5690 consensus_queue.cc:260] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:53.936739 5416 heartbeater.cc:503] Master 127.4.40.62:45809 was elected leader, sending a full tablet report...
I20260811 07:00:53.937302 5690 ts_tablet_manager.cc:1428] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260811 07:00:53.968983 5182 tablet.cc:2369] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:00:54.012234 5417 tablet.cc:2369] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:00:54.054870 5657 tablet.cc:2369] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:54.269340 5700 raft_consensus.cc:482] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:54.269665 5700 raft_consensus.cc:504] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:54.271282 5700 leader_election.cc:288] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 44ff4cd489ab4a41aee2090c2b58f84c (127.4.40.3:35699), 0f421251d0c4490095d22a11a78d806a (127.4.40.1:35785)
I20260811 07:00:54.278465 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.271757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:52041 (local address 127.4.40.1:35785)
0811 07:00:54.271979 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:54.271985 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:54.272981 (+ 996us) server_negotiation.cc:407] Connection header received
0811 07:00:54.273106 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:54.273111 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:54.273190 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:54.273284 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:54.273891 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.274739 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.276646 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.277424 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.277537 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.277657 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:54.278051 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 07:00:54.278193 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 07:00:54.278507 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.271816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:50495 (local address 127.4.40.3:35699)
0811 07:00:54.272038 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:54.272045 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:54.273124 (+ 1079us) server_negotiation.cc:407] Connection header received
0811 07:00:54.273291 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:54.273296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:54.273374 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:54.273479 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:54.274117 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.274852 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.276694 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.277480 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.277653 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.277752 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:54.278116 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 07:00:54.278266 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 07:00:54.278532 5701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.271853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:35785 (local address 127.4.40.2:52041)
0811 07:00:54.272810 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:54.272851 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:54.272982 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:54.273308 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:54.273313 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:54.273335 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:54.273730 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:54.273740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.274885 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.274894 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:54.276472 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:54.276485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.277664 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.277670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:54.277873 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.277906 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 07:00:54.277979 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:00:54.278102 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":6,"thread_start_us":431,"threads_started":1}
I20260811 07:00:54.278673 5703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.271853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:35699 (local address 127.4.40.2:50495)
0811 07:00:54.273003 (+ 1150us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:54.273022 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:54.273150 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:54.273518 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:54.273521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:54.273541 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:54.273897 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:54.273926 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.274990 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.274996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:54.276502 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:54.276514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.277754 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.277758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:54.277925 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.277947 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 07:00:54.278068 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 07:00:54.278192 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":263,"thread_start_us":566,"threads_started":1}
I20260811 07:00:54.279649 5576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5560b20860b94035a867c5cecf6cb55f" candidate_uuid: "864cced67bda44d989ad27b88f7ff3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" is_pre_election: true
I20260811 07:00:54.279639 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5560b20860b94035a867c5cecf6cb55f" candidate_uuid: "864cced67bda44d989ad27b88f7ff3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f421251d0c4490095d22a11a78d806a" is_pre_election: true
I20260811 07:00:54.280144 5576 raft_consensus.cc:2454] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 864cced67bda44d989ad27b88f7ff3a0 in term 0.
I20260811 07:00:54.280164 5106 raft_consensus.cc:2454] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 864cced67bda44d989ad27b88f7ff3a0 in term 0.
I20260811 07:00:54.280776 5226 leader_election.cc:302] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f421251d0c4490095d22a11a78d806a, 864cced67bda44d989ad27b88f7ff3a0; no voters:
I20260811 07:00:54.281122 5700 raft_consensus.cc:2781] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:00:54.281282 5700 raft_consensus.cc:482] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:00:54.281390 5700 raft_consensus.cc:3037] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:54.283555 5700 raft_consensus.cc:504] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:54.284181 5700 leader_election.cc:288] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [CANDIDATE]: Term 1 election: Requested vote from peers 44ff4cd489ab4a41aee2090c2b58f84c (127.4.40.3:35699), 0f421251d0c4490095d22a11a78d806a (127.4.40.1:35785)
I20260811 07:00:54.284602 5576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5560b20860b94035a867c5cecf6cb55f" candidate_uuid: "864cced67bda44d989ad27b88f7ff3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "44ff4cd489ab4a41aee2090c2b58f84c"
I20260811 07:00:54.284621 5106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5560b20860b94035a867c5cecf6cb55f" candidate_uuid: "864cced67bda44d989ad27b88f7ff3a0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f421251d0c4490095d22a11a78d806a"
I20260811 07:00:54.284799 5576 raft_consensus.cc:3037] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:54.284803 5106 raft_consensus.cc:3037] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:54.286897 5576 raft_consensus.cc:2454] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 864cced67bda44d989ad27b88f7ff3a0 in term 1.
I20260811 07:00:54.286975 5106 raft_consensus.cc:2454] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 864cced67bda44d989ad27b88f7ff3a0 in term 1.
I20260811 07:00:54.287285 5227 leader_election.cc:302] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44ff4cd489ab4a41aee2090c2b58f84c, 864cced67bda44d989ad27b88f7ff3a0; no voters:
I20260811 07:00:54.287606 5700 raft_consensus.cc:2781] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:54.287998 5700 raft_consensus.cc:686] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 1 LEADER]: Becoming Leader. State: Replica: 864cced67bda44d989ad27b88f7ff3a0, State: Running, Role: LEADER
I20260811 07:00:54.288530 5700 consensus_queue.cc:237] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } }
I20260811 07:00:54.292831 4809 catalog_manager.cc:5495] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 864cced67bda44d989ad27b88f7ff3a0 (127.4.40.2). New cstate: current_term: 1 leader_uuid: "864cced67bda44d989ad27b88f7ff3a0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "864cced67bda44d989ad27b88f7ff3a0" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 35109 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 } health_report { overall_health: UNKNOWN } } }
I20260811 07:00:54.346330 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.338814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44584 (local address 127.4.40.2:35109)
0811 07:00:54.339399 (+ 585us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:54.339408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:54.339429 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:00:54.339553 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:54.339560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:54.339729 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:54.339900 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:00:54.340458 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.341280 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.343685 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.344084 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.344277 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.345040 (+ 763us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:00:54.345655 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:00:54.345788 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:54.346022 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 07:00:54.346071 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395}
I20260811 07:00:54.356499 5106 raft_consensus.cc:1261] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 1 FOLLOWER]: Refusing update from remote peer 864cced67bda44d989ad27b88f7ff3a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:54.356706 5576 raft_consensus.cc:1261] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 1 FOLLOWER]: Refusing update from remote peer 864cced67bda44d989ad27b88f7ff3a0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:54.357575 5704 consensus_queue.cc:1035] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f421251d0c4490095d22a11a78d806a" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 35785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:54.357970 5700 consensus_queue.cc:1035] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "44ff4cd489ab4a41aee2090c2b58f84c" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 35699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:54.386857 4809 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:54.387122 4809 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:54.390597 4809 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0183910eec014136a1ba45da7902b905: Sending DeleteTablet for 3 replicas of tablet 5560b20860b94035a867c5cecf6cb55f
I20260811 07:00:54.391916 5305 tablet_service.cc:1514] Processing DeleteTablet for tablet 5560b20860b94035a867c5cecf6cb55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:44574
I20260811 07:00:54.391871 5545 tablet_service.cc:1514] Processing DeleteTablet for tablet 5560b20860b94035a867c5cecf6cb55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:38238
I20260811 07:00:54.391919 5061 tablet_service.cc:1514] Processing DeleteTablet for tablet 5560b20860b94035a867c5cecf6cb55f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:54 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:43644
I20260811 07:00:54.394263 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4929
I20260811 07:00:54.407400 5728 tablet_replica.cc:330] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: stopping tablet replica
I20260811 07:00:54.407915 5728 raft_consensus.cc:2229] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:54.408622 5728 raft_consensus.cc:2258] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:54.411031 5728 ts_tablet_manager.cc:1905] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:54.419992 5728 ts_tablet_manager.cc:1918] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:54.420298 5728 log.cc:1196] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/5560b20860b94035a867c5cecf6cb55f
I20260811 07:00:54.420982 5728 ts_tablet_manager.cc:1939] T 5560b20860b94035a867c5cecf6cb55f P 0f421251d0c4490095d22a11a78d806a: Deleting consensus metadata
I20260811 07:00:54.423023 4785 catalog_manager.cc:4841] TS 0f421251d0c4490095d22a11a78d806a (127.4.40.1:35785): tablet 5560b20860b94035a867c5cecf6cb55f (table test-table [id=c217dc9c7476474bad165117ecb4701b]) successfully deleted
I20260811 07:00:54.424651 5726 tablet_replica.cc:330] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: stopping tablet replica
I20260811 07:00:54.425161 5726 raft_consensus.cc:2229] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:54.425807 5726 raft_consensus.cc:2258] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:54.428004 5726 ts_tablet_manager.cc:1905] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:54.428761 5725 tablet_replica.cc:330] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: stopping tablet replica
I20260811 07:00:54.429205 5725 raft_consensus.cc:2229] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:00:54.429894 5725 raft_consensus.cc:2258] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 07:00:54 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 07:00:54.433400 5725 ts_tablet_manager.cc:1905] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:54.437438 5726 ts_tablet_manager.cc:1918] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:54.437716 5726 log.cc:1196] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/5560b20860b94035a867c5cecf6cb55f
I20260811 07:00:54.438402 5726 ts_tablet_manager.cc:1939] T 5560b20860b94035a867c5cecf6cb55f P 44ff4cd489ab4a41aee2090c2b58f84c: Deleting consensus metadata
I20260811 07:00:54.440682 4786 catalog_manager.cc:4841] TS 44ff4cd489ab4a41aee2090c2b58f84c (127.4.40.3:35699): tablet 5560b20860b94035a867c5cecf6cb55f (table test-table [id=c217dc9c7476474bad165117ecb4701b]) successfully deleted
I20260811 07:00:54.442207 5725 ts_tablet_manager.cc:1918] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:54.442395 5725 log.cc:1196] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/5560b20860b94035a867c5cecf6cb55f
I20260811 07:00:54.442675 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.396238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.4.40.1:35785)
0811 07:00:54.396463 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:54.396471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:54.421598 (+ 25127us) server_negotiation.cc:407] Connection header received
0811 07:00:54.422127 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:54.422134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:54.422224 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:54.422302 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:54.424227 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.425334 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.427270 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.427873 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.427971 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.433779 (+ 5808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:54.433794 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:54.433810 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:54.433852 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:54.438467 (+ 4615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:54.439472 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:54.439480 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:54.439484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:54.439699 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:54.440284 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:54.440291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:54.440294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:54.440750 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:54.441100 (+ 350us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:54.441757 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 07:00:54.442014 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 07:00:54.442998 5725 ts_tablet_manager.cc:1939] T 5560b20860b94035a867c5cecf6cb55f P 864cced67bda44d989ad27b88f7ff3a0: Deleting consensus metadata
I20260811 07:00:54.443409 5171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:54.445673 4778 catalog_manager.cc:4841] TS 864cced67bda44d989ad27b88f7ff3a0 (127.4.40.2:35109): tablet 5560b20860b94035a867c5cecf6cb55f (table test-table [id=c217dc9c7476474bad165117ecb4701b]) successfully deleted
I20260811 07:00:54.550391 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4929
I20260811 07:00:54.585124 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5187
Aug 11 07:00:54 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 07:00:54.603636 5685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.585927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44588 (local address 127.4.40.2:35109)
0811 07:00:54.586152 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:54.586158 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:54.587532 (+ 1374us) server_negotiation.cc:407] Connection header received
0811 07:00:54.587663 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:54.587668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:54.587766 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:54.587890 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:54.589269 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.590516 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.591925 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.592710 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.592842 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.597287 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:54.597318 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:54.597343 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:54.597393 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:54.600829 (+ 3436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:54.601513 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:54.601524 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:54.601526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:54.601670 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:54.602059 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:54.602064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:54.602072 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:54.602412 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:54.602833 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:54.603030 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 07:00:54.603260 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:00:54.604346 5410 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:54.718086 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5187
I20260811 07:00:54.751024 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5428
Aug 11 07:00:54 dist-test-slave-7vjv krb5kdc[4724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 07:00:54.765774 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:54.751863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38252 (local address 127.4.40.3:35699)
0811 07:00:54.752036 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:54.752041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:54.752059 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:00:54.752268 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:54.752272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:54.752368 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:54.752451 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:54.753718 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.754435 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.755629 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:54.756012 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:54.756160 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:54.760650 (+ 4490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:54.760662 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:54.760678 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:00:54.760715 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:54.763246 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:54.763871 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:54.763878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:54.763881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:54.764020 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:00:54.764401 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:00:54.764404 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:00:54.764406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:00:54.764771 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:54.764978 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:54.765241 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 07:00:54.765397 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:00:54.766451 5646 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:54.854387 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5428
I20260811 07:00:54.881765 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 4743
I20260811 07:00:54.882892 4866 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:54.993137 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 4743
2026-08-11T07:00:55Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (5110 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T07:00:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:00:55Z Disabled control of system clock
I20260811 07:00:55.095841 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:33281
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:33281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:00:55.447379 5742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:55.447670 5742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:55.447768 5742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:55.455118 5742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:00:55.455229 5742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:55.455282 5742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:55.455348 5742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:00:55.455399 5742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:00:55.464241 5742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:33281
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:33281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:55.465389 5742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:55.467085 5742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:55.478464 5749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:55.478681 5751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:55.478770 5753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:55.479802 5742 server_base.cc:1034] running on GCE node
I20260811 07:00:55.480597 5742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:55.481994 5742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:55.483225 5742 hybrid_clock.cc:648] HybridClock initialized: now 1786431655483186 us; error 69 us; skew 500 ppm
I20260811 07:00:55.483575 5742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:55.485843 5742 webserver.cc:460] Webserver started at http://127.4.40.62:34735/ using document root <none> and password file <none>
I20260811 07:00:55.486529 5742 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:55.486658 5742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:55.486972 5742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:55.489957 5742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "fb7fbfe964ad4d28945f8a69bfe9054f"
format_stamp: "Formatted at 2026-08-11 07:00:55 on dist-test-slave-7vjv"
server_key: "1b28460ef599c4ae2f00d4de4ed1677c"
server_key_iv: "b8fb8f1b7768c873a8de4a9ee75c35f7"
server_key_version: "encryptionkey@0"
I20260811 07:00:55.490775 5742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "fb7fbfe964ad4d28945f8a69bfe9054f"
format_stamp: "Formatted at 2026-08-11 07:00:55 on dist-test-slave-7vjv"
server_key: "1b28460ef599c4ae2f00d4de4ed1677c"
server_key_iv: "b8fb8f1b7768c873a8de4a9ee75c35f7"
server_key_version: "encryptionkey@0"
I20260811 07:00:55.495817 5742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 07:00:55.499150 5769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:55.500528 5742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:55.500718 5742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "fb7fbfe964ad4d28945f8a69bfe9054f"
format_stamp: "Formatted at 2026-08-11 07:00:55 on dist-test-slave-7vjv"
server_key: "1b28460ef599c4ae2f00d4de4ed1677c"
server_key_iv: "b8fb8f1b7768c873a8de4a9ee75c35f7"
server_key_version: "encryptionkey@0"
I20260811 07:00:55.500965 5742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:55.520061 5742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:55.520952 5742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:55.521271 5742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:55.540612 5742 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:33281
I20260811 07:00:55.540637 5855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:33281 every 8 connection(s)
I20260811 07:00:55.541956 5742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:00:55.542438 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5742
I20260811 07:00:55.542644 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:00:55.543275 4256 external_mini_cluster.cc:1428] Setting key 31026c24dfb3ee84052afef464fb4d56
I20260811 07:00:55.546483 5857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:55.556522 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:55.545275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40942 (local address 127.4.40.62:33281)
0811 07:00:55.546250 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:55.546261 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:55.546301 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:00:55.547988 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:55.548010 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:55.548487 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:55.548871 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:55.549463 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:55.550735 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:55.552073 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:55.552717 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:55.552931 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:55.554227 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:55.554234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:55.554247 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:55.554794 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:55.554866 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:55.554893 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:55.555034 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:55.555278 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 07:00:55.555574 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":358,"threads_started":1}
I20260811 07:00:55.557694 5857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f: Bootstrap starting.
I20260811 07:00:55.561115 5857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:55.562135 5857 log.cc:824] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:55.564775 5857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f: No bootstrap required, opened a new log
I20260811 07:00:55.571558 5857 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33281 } }
I20260811 07:00:55.571897 5857 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:55.572021 5857 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb7fbfe964ad4d28945f8a69bfe9054f, State: Initialized, Role: FOLLOWER
I20260811 07:00:55.572592 5857 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33281 } }
I20260811 07:00:55.572822 5857 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:00:55.572999 5857 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:00:55.573167 5857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:55.575212 5857 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33281 } }
I20260811 07:00:55.575696 5857 leader_election.cc:302] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb7fbfe964ad4d28945f8a69bfe9054f; no voters:
I20260811 07:00:55.576562 5857 leader_election.cc:288] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:00:55.576675 5867 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:55.577524 5867 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [term 1 LEADER]: Becoming Leader. State: Replica: fb7fbfe964ad4d28945f8a69bfe9054f, State: Running, Role: LEADER
I20260811 07:00:55.578107 5867 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33281 } }
I20260811 07:00:55.578656 5857 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:00:55.580684 5869 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb7fbfe964ad4d28945f8a69bfe9054f. Latest consensus state: current_term: 1 leader_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33281 } } }
I20260811 07:00:55.581096 5869 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:55.581257 5868 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb7fbfe964ad4d28945f8a69bfe9054f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 33281 } } }
I20260811 07:00:55.581773 5868 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f [sys.catalog]: This master's current role is: LEADER
I20260811 07:00:55.584225 5873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:00:55.589836 5873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:00:55.598522 5873 catalog_manager.cc:1347] Generated new cluster ID: f7514958f4004c3fa70f186af3cc9ef2
I20260811 07:00:55.598683 5873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:00:55.609323 5873 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:00:55.610742 5873 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:00:55.622900 5873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f: Generated new TSK 0
I20260811 07:00:55.623620 5873 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:00:55.638219 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33281
--builtin_ntp_servers=127.4.40.20:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:00:55.986737 5900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:55.986999 5900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:55.987156 5900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:55.993718 5900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:55.993801 5900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:55.993928 5900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:00:56.002491 5900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:56.003687 5900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:56.005757 5900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:56.018929 5911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:56.018982 5914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:56.019026 5909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:56.025180 5900 server_base.cc:1034] running on GCE node
I20260811 07:00:56.025836 5900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:56.026733 5900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:56.027931 5900 hybrid_clock.cc:648] HybridClock initialized: now 1786431656027871 us; error 57 us; skew 500 ppm
I20260811 07:00:56.028261 5900 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:56.030391 5900 webserver.cc:460] Webserver started at http://127.4.40.1:39657/ using document root <none> and password file <none>
I20260811 07:00:56.031002 5900 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:56.031086 5900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:56.031344 5900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:56.034145 5900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "29edc79c41d24dcbb1f93a95e3fa16ff"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "04aeb8249fc942c5c23a1e03852b9cc2"
server_key_iv: "9e5c42fef646b2320fb3f3439d8fd595"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.034883 5900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "29edc79c41d24dcbb1f93a95e3fa16ff"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "04aeb8249fc942c5c23a1e03852b9cc2"
server_key_iv: "9e5c42fef646b2320fb3f3439d8fd595"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.039779 5900 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 07:00:56.043159 5920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:56.044484 5900 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:00:56.044613 5900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "29edc79c41d24dcbb1f93a95e3fa16ff"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "04aeb8249fc942c5c23a1e03852b9cc2"
server_key_iv: "9e5c42fef646b2320fb3f3439d8fd595"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.044785 5900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:56.069597 5900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:56.070325 5900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:56.070571 5900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:56.071465 5900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:56.073019 5900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:56.073132 5900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:56.073267 5900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:56.073375 5900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:56.108258 5900 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:42117
I20260811 07:00:56.108368 6124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:42117 every 8 connection(s)
I20260811 07:00:56.109606 5900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:00:56.112493 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 5900
I20260811 07:00:56.112665 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 07:00:56.113265 4256 external_mini_cluster.cc:1428] Setting key 2e84920eb5e368efe8103429af01b6e8
I20260811 07:00:56.116175 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33281
--builtin_ntp_servers=127.4.40.20:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:00:56.123889 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:56.112173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:45833 (local address 127.4.40.62:33281)
0811 07:00:56.112712 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:56.112720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:56.114185 (+ 1465us) server_negotiation.cc:407] Connection header received
0811 07:00:56.116070 (+ 1885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:56.116075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:56.116189 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:56.116332 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:56.117713 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.118417 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.120876 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.121279 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.121385 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:56.123114 (+ 1729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:56.123126 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:56.123132 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:56.123162 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:56.123221 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:56.123235 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:56.123307 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:56.123560 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 07:00:56.123634 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":266,"threads_started":1}
I20260811 07:00:56.125331 6133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:56.112833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33281 (local address 127.4.40.1:45833)
0811 07:00:56.113897 (+ 1064us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:56.113946 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:56.115800 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:56.116586 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:56.116599 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:56.116716 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:56.117532 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:56.117560 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.118582 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.118590 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:56.120734 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:56.120751 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.121532 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.121542 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:56.121874 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:56.122683 (+ 809us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:00:56.122689 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:56.122742 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:00:56.122842 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:56.123340 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:56.123355 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:56.123366 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 07:00:56.123543 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 07:00:56.123942 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":346,"threads_started":1}
I20260811 07:00:56.126834 6128 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33281
I20260811 07:00:56.127171 6128 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:56.127885 6128 heartbeater.cc:511] Master 127.4.40.62:33281 requested a full tablet report, sending...
I20260811 07:00:56.130260 5796 ts_manager.cc:194] Registered new tserver with Master: 29edc79c41d24dcbb1f93a95e3fa16ff (127.4.40.1:42117)
I20260811 07:00:56.131824 5796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.1:45833
W20260811 07:00:56.477255 6135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:56.477579 6135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:56.477711 6135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:56.485692 6135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:56.485811 6135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:56.486017 6135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:00:56.494966 6135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:56.496415 6135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:56.498368 6135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:56.512310 6147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:56.512374 6141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:56.512398 6140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:56.513077 6135 server_base.cc:1034] running on GCE node
I20260811 07:00:56.513790 6135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:56.514798 6135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:56.516125 6135 hybrid_clock.cc:648] HybridClock initialized: now 1786431656515994 us; error 56 us; skew 500 ppm
I20260811 07:00:56.516491 6135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:56.518888 6135 webserver.cc:460] Webserver started at http://127.4.40.2:33771/ using document root <none> and password file <none>
I20260811 07:00:56.519670 6135 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:56.519820 6135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:56.520318 6135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:56.523360 6135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "d75e24e2145540bf893c6234d519d124"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "20d087c071adab6228ac2e46a7a71074"
server_key_iv: "bac1d1bc7913f98bfb10c40b21e54802"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.524317 6135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "d75e24e2145540bf893c6234d519d124"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "20d087c071adab6228ac2e46a7a71074"
server_key_iv: "bac1d1bc7913f98bfb10c40b21e54802"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.529611 6135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 07:00:56.533165 6160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:56.534441 6135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:00:56.534677 6135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "d75e24e2145540bf893c6234d519d124"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "20d087c071adab6228ac2e46a7a71074"
server_key_iv: "bac1d1bc7913f98bfb10c40b21e54802"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.534920 6135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:56.553363 6135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:56.554162 6135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:56.554471 6135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:56.555691 6135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:56.557286 6135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:56.557446 6135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:56.557587 6135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:56.557688 6135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:56.593176 6135 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:44775
I20260811 07:00:56.593228 6375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:44775 every 8 connection(s)
I20260811 07:00:56.594547 6135 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:00:56.602062 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6135
I20260811 07:00:56.602238 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 07:00:56.602842 4256 external_mini_cluster.cc:1428] Setting key 0afaadea5b8781480286046c8d8d3a5e
I20260811 07:00:56.606006 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33281
--builtin_ntp_servers=127.4.40.20:33047
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:00:56.607528 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:56.597085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:57379 (local address 127.4.40.62:33281)
0811 07:00:56.597330 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:56.597338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:56.598707 (+ 1369us) server_negotiation.cc:407] Connection header received
0811 07:00:56.600490 (+ 1783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:56.600495 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:56.600559 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:56.600649 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:56.601646 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.602465 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.604508 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.605083 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.605181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:56.606789 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:56.606794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:56.606797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:56.606814 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:56.606843 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:56.606854 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:56.606919 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:56.607233 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 07:00:56.607303 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 07:00:56.608953 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:56.597626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33281 (local address 127.4.40.2:57379)
0811 07:00:56.598553 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:56.598596 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:56.600229 (+ 1633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:56.600880 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:56.600887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:56.600975 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:56.601504 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:56.601519 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.602621 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.602626 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:56.604372 (+ 1746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:56.604384 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:56.605317 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:56.605324 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:56.605618 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:56.606448 (+ 830us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:00:56.606454 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:56.606502 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:00:56.606593 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:56.606952 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:56.606972 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:56.606984 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 07:00:56.607220 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 07:00:56.607621 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":314,"threads_started":1}
I20260811 07:00:56.610357 6376 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33281
I20260811 07:00:56.610656 6376 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:56.611574 6376 heartbeater.cc:511] Master 127.4.40.62:33281 requested a full tablet report, sending...
I20260811 07:00:56.613073 5796 ts_manager.cc:194] Registered new tserver with Master: d75e24e2145540bf893c6234d519d124 (127.4.40.2:44775)
I20260811 07:00:56.613976 5796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.2:57379
W20260811 07:00:56.949833 6383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:00:56.950093 6383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:00:56.950268 6383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:00:56.957357 6383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:00:56.957466 6383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:00:56.957639 6383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:00:56.966341 6383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33047
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:33281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:00:56.967626 6383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:00:56.969465 6383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:00:56.982383 6391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:56.982422 6390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:00:56.982342 6394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:00:56.983338 6383 server_base.cc:1034] running on GCE node
I20260811 07:00:56.983987 6383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:00:56.985071 6383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:00:56.986296 6383 hybrid_clock.cc:648] HybridClock initialized: now 1786431656986221 us; error 47 us; skew 500 ppm
I20260811 07:00:56.986581 6383 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:00:56.991663 6383 webserver.cc:460] Webserver started at http://127.4.40.3:45815/ using document root <none> and password file <none>
I20260811 07:00:56.992370 6383 fs_manager.cc:359] Metadata directory not provided
I20260811 07:00:56.992460 6383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:00:56.992738 6383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:00:56.995533 6383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "7453613c1cac4b5d90a8edf94074ff07"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "7e6d86c23e6a2016876df9b13f0203c2"
server_key_iv: "3f4a2a1d39deb7da6607ae3fec35ec09"
server_key_version: "encryptionkey@0"
I20260811 07:00:56.996348 6383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "7453613c1cac4b5d90a8edf94074ff07"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "7e6d86c23e6a2016876df9b13f0203c2"
server_key_iv: "3f4a2a1d39deb7da6607ae3fec35ec09"
server_key_version: "encryptionkey@0"
I20260811 07:00:57.001133 6383 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 07:00:57.004491 6412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:57.005908 6383 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 07:00:57.006048 6383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "7453613c1cac4b5d90a8edf94074ff07"
format_stamp: "Formatted at 2026-08-11 07:00:56 on dist-test-slave-7vjv"
server_key: "7e6d86c23e6a2016876df9b13f0203c2"
server_key_iv: "3f4a2a1d39deb7da6607ae3fec35ec09"
server_key_version: "encryptionkey@0"
I20260811 07:00:57.006212 6383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:00:57.025990 6383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:00:57.026815 6383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:00:57.027094 6383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:00:57.028025 6383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:00:57.029503 6383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:00:57.029627 6383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:57.029755 6383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:00:57.029847 6383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:00:57.063990 6383 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:44067
I20260811 07:00:57.064030 6609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:44067 every 8 connection(s)
I20260811 07:00:57.065341 6383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:00:57.071161 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6383
I20260811 07:00:57.071365 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 07:00:57.071924 4256 external_mini_cluster.cc:1428] Setting key 5447ace814400a3cad47d39b152829e8
I20260811 07:00:57.077924 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.067770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:35081 (local address 127.4.40.62:33281)
0811 07:00:57.067940 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.067944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.069303 (+ 1359us) server_negotiation.cc:407] Connection header received
0811 07:00:57.071091 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.071095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.071162 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.071235 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:57.072629 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.073642 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.075467 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.075864 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.075969 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.077275 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:57.077280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:57.077283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:57.077300 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:57.077331 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:57.077343 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:57.077415 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.077621 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.077695 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:00:57.078931 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.068298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:33281 (local address 127.4.40.3:35081)
0811 07:00:57.069158 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.069191 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:57.070800 (+ 1609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:57.071505 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:57.071517 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:57.071620 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:00:57.072442 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.072469 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.073845 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.073850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.075339 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.075351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.076088 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.076114 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.076368 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.076995 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:00:57.076998 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:00:57.077034 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:00:57.077108 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:00:57.077427 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:00:57.077435 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:00:57.077440 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 07:00:57.077576 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 07:00:57.077842 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":277,"threads_started":1}
I20260811 07:00:57.080245 6610 heartbeater.cc:347] Connected to a master server at 127.4.40.62:33281
I20260811 07:00:57.080536 6610 heartbeater.cc:464] Registering TS with master...
I20260811 07:00:57.081261 6610 heartbeater.cc:511] Master 127.4.40.62:33281 requested a full tablet report, sending...
I20260811 07:00:57.082551 5796 ts_manager.cc:194] Registered new tserver with Master: 7453613c1cac4b5d90a8edf94074ff07 (127.4.40.3:44067)
I20260811 07:00:57.083325 5796 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.3:35081
I20260811 07:00:57.086972 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:00:57.096524 6132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.091039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.4.40.62:33281)
0811 07:00:57.091189 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.091193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.091614 (+ 421us) server_negotiation.cc:407] Connection header received
0811 07:00:57.091737 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.091741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.091803 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.091874 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:57.092367 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.093009 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.094315 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.094728 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.094821 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.095785 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:57.095789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:57.095792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:57.095808 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:57.095836 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:57.095847 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:57.095964 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.096188 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.096229 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:00:57.099938 5796 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:00:57.106860 5796 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:00:57.135289 6128 heartbeater.cc:503] Master 127.4.40.62:33281 was elected leader, sending a full tablet report...
I20260811 07:00:57.140215 6629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.119335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43828)
0811 07:00:57.130429 (+ 11094us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.130476 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:57.130655 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:57.132009 (+ 1354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:57.132015 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:57.132045 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:57.132551 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.132566 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.134572 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.134578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.137083 (+ 2505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.137098 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.139155 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.139161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.139376 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.139434 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 07:00:57.139559 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 07:00:57.139783 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10853,"mutex_wait_us":18,"thread_start_us":10704,"threads_started":1}
I20260811 07:00:57.140226 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.119570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43828 (local address 127.4.40.1:42117)
0811 07:00:57.131086 (+ 11516us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.131097 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.131132 (+ 35us) server_negotiation.cc:407] Connection header received
0811 07:00:57.131235 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.131244 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.131516 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.131705 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:57.132786 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.134384 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.137322 (+ 2938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.138897 (+ 1575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.139068 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.139354 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.139675 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.139841 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11318,"thread_start_us":11244,"threads_started":1}
I20260811 07:00:57.141103 6635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.119708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44067 (local address 127.0.0.1:59876)
0811 07:00:57.131422 (+ 11714us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.131439 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:57.131547 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:57.133038 (+ 1491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:57.133043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:57.133068 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:57.133443 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.133457 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.135812 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.135829 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.138113 (+ 2284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.138129 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.140255 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.140263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.140437 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.140475 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 07:00:57.140576 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 07:00:57.140695 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11597,"mutex_wait_us":37,"thread_start_us":17814,"threads_started":1}
I20260811 07:00:57.142306 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.120293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59876 (local address 127.4.40.3:44067)
0811 07:00:57.132156 (+ 11863us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.132166 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.132199 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:00:57.132294 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.132301 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.132547 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.132742 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:57.133648 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.135641 (+ 1993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.138374 (+ 2733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.139901 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.140047 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.141724 (+ 1677us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.141798 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.141979 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11651,"thread_start_us":11508,"threads_started":1}
I20260811 07:00:57.144544 6013 tablet_service.cc:1467] Processing CreateTablet for tablet fc44ea5447794a868cae74de6bc8349a (DEFAULT_TABLE table=test-table [id=f212e41d44164e27986fed567c46928b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:57.145903 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.120064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44775 (local address 127.0.0.1:44606)
0811 07:00:57.137418 (+ 17354us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.137434 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:57.137837 (+ 403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:57.138596 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:57.138600 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:57.138621 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:57.138949 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.138959 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.140958 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.140964 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.142748 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.142761 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.145178 (+ 2417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.145186 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.145360 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.145390 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 07:00:57.145485 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 07:00:57.145610 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17239,"thread_start_us":11073,"threads_started":1}
I20260811 07:00:57.146347 6013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc44ea5447794a868cae74de6bc8349a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:57.146374 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.120593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44606 (local address 127.4.40.2:44775)
0811 07:00:57.136981 (+ 16388us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.136989 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.137740 (+ 751us) server_negotiation.cc:407] Connection header received
0811 07:00:57.138058 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.138064 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.138275 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.138442 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:57.139203 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.140781 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.142928 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.144410 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.144792 (+ 382us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.145036 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.145868 (+ 832us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.146040 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16228,"thread_start_us":18022,"threads_started":1}
I20260811 07:00:57.146030 6500 tablet_service.cc:1467] Processing CreateTablet for tablet fc44ea5447794a868cae74de6bc8349a (DEFAULT_TABLE table=test-table [id=f212e41d44164e27986fed567c46928b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:57.147311 6500 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc44ea5447794a868cae74de6bc8349a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:57.149909 6250 tablet_service.cc:1467] Processing CreateTablet for tablet fc44ea5447794a868cae74de6bc8349a (DEFAULT_TABLE table=test-table [id=f212e41d44164e27986fed567c46928b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:00:57.151244 6250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc44ea5447794a868cae74de6bc8349a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:00:57.158587 6638 tablet_bootstrap.cc:492] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Bootstrap starting.
I20260811 07:00:57.161626 6641 tablet_bootstrap.cc:492] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Bootstrap starting.
I20260811 07:00:57.162545 6638 tablet_bootstrap.cc:654] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:57.163825 6638 log.cc:824] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:57.165354 6641 tablet_bootstrap.cc:654] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:57.166921 6641 log.cc:824] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:57.170914 6642 tablet_bootstrap.cc:492] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Bootstrap starting.
I20260811 07:00:57.172497 6641 tablet_bootstrap.cc:492] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: No bootstrap required, opened a new log
I20260811 07:00:57.172761 6641 ts_tablet_manager.cc:1397] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20260811 07:00:57.174003 6642 tablet_bootstrap.cc:654] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Neither blocks nor log segments found. Creating new log.
I20260811 07:00:57.174690 6638 tablet_bootstrap.cc:492] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: No bootstrap required, opened a new log
I20260811 07:00:57.174983 6638 ts_tablet_manager.cc:1397] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.004s
I20260811 07:00:57.175007 6642 log.cc:824] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Log is configured to *not* fsync() on all Append() calls
I20260811 07:00:57.177977 6642 tablet_bootstrap.cc:492] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: No bootstrap required, opened a new log
I20260811 07:00:57.178278 6642 ts_tablet_manager.cc:1397] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 07:00:57.183348 6641 raft_consensus.cc:348] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.183777 6641 raft_consensus.cc:374] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:57.183876 6641 raft_consensus.cc:729] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7453613c1cac4b5d90a8edf94074ff07, State: Initialized, Role: FOLLOWER
I20260811 07:00:57.184526 6641 consensus_queue.cc:260] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.185290 6638 raft_consensus.cc:348] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.185614 6610 heartbeater.cc:503] Master 127.4.40.62:33281 was elected leader, sending a full tablet report...
I20260811 07:00:57.185703 6638 raft_consensus.cc:374] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:57.185794 6638 raft_consensus.cc:729] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29edc79c41d24dcbb1f93a95e3fa16ff, State: Initialized, Role: FOLLOWER
I20260811 07:00:57.185847 6642 raft_consensus.cc:348] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.186204 6642 raft_consensus.cc:374] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:00:57.186311 6642 raft_consensus.cc:729] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d75e24e2145540bf893c6234d519d124, State: Initialized, Role: FOLLOWER
I20260811 07:00:57.186394 6638 consensus_queue.cc:260] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.186828 6642 consensus_queue.cc:260] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.188150 6638 ts_tablet_manager.cc:1428] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 07:00:57.188129 6641 ts_tablet_manager.cc:1428] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Time spent starting tablet: real 0.015s user 0.013s sys 0.001s
I20260811 07:00:57.190546 6376 heartbeater.cc:503] Master 127.4.40.62:33281 was elected leader, sending a full tablet report...
I20260811 07:00:57.191270 6642 ts_tablet_manager.cc:1428] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 07:00:57.279219 6650 raft_consensus.cc:482] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:00:57.279609 6650 raft_consensus.cc:504] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.281453 6650 leader_election.cc:288] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 29edc79c41d24dcbb1f93a95e3fa16ff (127.4.40.1:42117), d75e24e2145540bf893c6234d519d124 (127.4.40.2:44775)
I20260811 07:00:57.288798 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.281809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:34725 (local address 127.4.40.1:42117)
0811 07:00:57.282043 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.282050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.282364 (+ 314us) server_negotiation.cc:407] Connection header received
0811 07:00:57.282600 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.282606 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.282689 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.282817 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:57.283267 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.284291 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.286764 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.287891 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.288022 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.288149 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.288389 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.288529 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:00:57.288837 6613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.281854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.4.40.3:34725)
0811 07:00:57.282222 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.282249 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:57.282408 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:57.282877 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:57.282883 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:57.282909 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:57.283095 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.283106 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.284414 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.284417 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.286613 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.286628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.288038 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.288042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.288215 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.288239 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 07:00:57.288349 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 07:00:57.288504 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":91}
I20260811 07:00:57.289129 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.281970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:42947 (local address 127.4.40.2:44775)
0811 07:00:57.282221 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.282230 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.282752 (+ 522us) server_negotiation.cc:407] Connection header received
0811 07:00:57.282879 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.282884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.282973 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.283071 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:00:57.283645 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.284633 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.287028 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.288191 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.288387 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.288513 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.288703 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.288830 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 07:00:57.289307 6653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.281885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:44775 (local address 127.4.40.3:42947)
0811 07:00:57.282645 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.282658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:00:57.282750 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:00:57.283115 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:00:57.283121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:00:57.283153 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:00:57.283471 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.283479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.284774 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.284779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.286853 (+ 2074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:00:57.286863 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.288369 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.288375 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:00:57.288532 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.288560 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 07:00:57.288648 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 07:00:57.288756 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":443,"threads_started":1}
I20260811 07:00:57.289927 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc44ea5447794a868cae74de6bc8349a" candidate_uuid: "7453613c1cac4b5d90a8edf94074ff07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" is_pre_election: true
I20260811 07:00:57.290361 6288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc44ea5447794a868cae74de6bc8349a" candidate_uuid: "7453613c1cac4b5d90a8edf94074ff07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75e24e2145540bf893c6234d519d124" is_pre_election: true
I20260811 07:00:57.290473 6043 raft_consensus.cc:2454] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7453613c1cac4b5d90a8edf94074ff07 in term 0.
I20260811 07:00:57.290836 6288 raft_consensus.cc:2454] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7453613c1cac4b5d90a8edf94074ff07 in term 0.
I20260811 07:00:57.291237 6415 leader_election.cc:302] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29edc79c41d24dcbb1f93a95e3fa16ff, 7453613c1cac4b5d90a8edf94074ff07; no voters:
I20260811 07:00:57.291687 6650 raft_consensus.cc:2781] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:00:57.291870 6650 raft_consensus.cc:482] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:00:57.291997 6650 raft_consensus.cc:3037] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:57.294313 6650 raft_consensus.cc:504] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.294899 6650 leader_election.cc:288] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [CANDIDATE]: Term 1 election: Requested vote from peers 29edc79c41d24dcbb1f93a95e3fa16ff (127.4.40.1:42117), d75e24e2145540bf893c6234d519d124 (127.4.40.2:44775)
I20260811 07:00:57.295351 6043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc44ea5447794a868cae74de6bc8349a" candidate_uuid: "7453613c1cac4b5d90a8edf94074ff07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff"
I20260811 07:00:57.295406 6288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc44ea5447794a868cae74de6bc8349a" candidate_uuid: "7453613c1cac4b5d90a8edf94074ff07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d75e24e2145540bf893c6234d519d124"
I20260811 07:00:57.295536 6043 raft_consensus.cc:3037] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:57.295670 6288 raft_consensus.cc:3037] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:00:57.297868 6043 raft_consensus.cc:2454] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7453613c1cac4b5d90a8edf94074ff07 in term 1.
I20260811 07:00:57.298023 6288 raft_consensus.cc:2454] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7453613c1cac4b5d90a8edf94074ff07 in term 1.
I20260811 07:00:57.298301 6415 leader_election.cc:302] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 29edc79c41d24dcbb1f93a95e3fa16ff, 7453613c1cac4b5d90a8edf94074ff07; no voters:
I20260811 07:00:57.298620 6650 raft_consensus.cc:2781] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:00:57.299088 6650 raft_consensus.cc:686] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 1 LEADER]: Becoming Leader. State: Replica: 7453613c1cac4b5d90a8edf94074ff07, State: Running, Role: LEADER
I20260811 07:00:57.299650 6650 consensus_queue.cc:237] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } }
I20260811 07:00:57.304337 5796 catalog_manager.cc:5495] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7453613c1cac4b5d90a8edf94074ff07 (127.4.40.3). New cstate: current_term: 1 leader_uuid: "7453613c1cac4b5d90a8edf94074ff07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7453613c1cac4b5d90a8edf94074ff07" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 } health_report { overall_health: UNKNOWN } } }
W20260811 07:00:57.316974 6612 tablet.cc:2369] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:57.341856 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.336160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59890 (local address 127.4.40.3:44067)
0811 07:00:57.336384 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.336390 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.336409 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:00:57.336580 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.336584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.336672 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.336828 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:00:57.337247 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.337949 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.339485 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.339963 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.340130 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.340765 (+ 635us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:00:57.341248 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:00:57.341441 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.341597 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.341638 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 07:00:57.346797 6379 tablet.cc:2369] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:57.350556 6288 raft_consensus.cc:1261] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 1 FOLLOWER]: Refusing update from remote peer 7453613c1cac4b5d90a8edf94074ff07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:57.350556 6043 raft_consensus.cc:1261] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 1 FOLLOWER]: Refusing update from remote peer 7453613c1cac4b5d90a8edf94074ff07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:00:57.351647 6654 consensus_queue.cc:1035] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d75e24e2145540bf893c6234d519d124" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 44775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:00:57.352030 6650 consensus_queue.cc:1035] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "29edc79c41d24dcbb1f93a95e3fa16ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 42117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 07:00:57.362172 6131 tablet.cc:2369] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:00:57.364539 6658 mvcc.cc:205] Tried to move back new op lower bound from 7317224068496777216 to 7317224068303749120. Current Snapshot: MvccSnapshot[applied={T|T < 7317224068496777216}]
I20260811 07:00:57.365741 6660 mvcc.cc:205] Tried to move back new op lower bound from 7317224068496777216 to 7317224068303749120. Current Snapshot: MvccSnapshot[applied={T|T < 7317224068496777216}]
I20260811 07:00:57.369705 6663 mvcc.cc:205] Tried to move back new op lower bound from 7317224068496777216 to 7317224068303749120. Current Snapshot: MvccSnapshot[applied={T|T < 7317224068496777216}]
I20260811 07:00:57.379412 5796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:57.379739 5796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:00:57.383862 5796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fb7fbfe964ad4d28945f8a69bfe9054f: Sending DeleteTablet for 3 replicas of tablet fc44ea5447794a868cae74de6bc8349a
I20260811 07:00:57.384918 6500 tablet_service.cc:1514] Processing DeleteTablet for tablet fc44ea5447794a868cae74de6bc8349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:57 UTC) from {username='slave'} at 127.0.0.1:59876
I20260811 07:00:57.384990 6013 tablet_service.cc:1514] Processing DeleteTablet for tablet fc44ea5447794a868cae74de6bc8349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:57 UTC) from {username='slave'} at 127.0.0.1:43828
I20260811 07:00:57.385928 6497 tablet_service.cc:1514] Processing DeleteTablet for tablet fc44ea5447794a868cae74de6bc8349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:57 UTC) from {username='slave'} at 127.0.0.1:59876
I20260811 07:00:57.387293 6250 tablet_service.cc:1514] Processing DeleteTablet for tablet fc44ea5447794a868cae74de6bc8349a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:00:57 UTC) from {username='slave'} at 127.0.0.1:44606
I20260811 07:00:57.387988 6681 tablet_replica.cc:330] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: stopping tablet replica
I20260811 07:00:57.388374 6682 tablet_replica.cc:330] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: stopping tablet replica
I20260811 07:00:57.388586 6681 raft_consensus.cc:2229] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:57.388914 6682 raft_consensus.cc:2229] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:00:57.389027 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5900
I20260811 07:00:57.389230 6681 raft_consensus.cc:2258] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:57.389427 6682 raft_consensus.cc:2258] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:57.390122 6680 tablet_replica.cc:330] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: stopping tablet replica
I20260811 07:00:57.390540 6680 raft_consensus.cc:2229] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:00:57.391162 6680 raft_consensus.cc:2258] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:00:57.393544 6682 ts_tablet_manager.cc:1905] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:57.393550 6680 ts_tablet_manager.cc:1905] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:57.395260 6681 ts_tablet_manager.cc:1905] T fc44ea5447794a868cae74de6bc8349a P 29edc79c41d24dcbb1f93a95e3fa16ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:00:57.399480 6631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.390443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43838 (local address 127.4.40.1:42117)
0811 07:00:57.390654 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.390660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.390968 (+ 308us) server_negotiation.cc:407] Connection header received
0811 07:00:57.391158 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.391164 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.391290 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.391407 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:57.392332 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.393492 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.395400 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.396055 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.396171 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.397700 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:57.397717 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:57.397731 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:57.398479 (+ 748us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:57.398549 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:57.398576 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:57.398675 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.398941 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.399061 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 07:00:57.400671 6682 ts_tablet_manager.cc:1918] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:57.400789 6118 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:57.400863 6682 log.cc:1196] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/fc44ea5447794a868cae74de6bc8349a
I20260811 07:00:57.401423 6682 ts_tablet_manager.cc:1939] T fc44ea5447794a868cae74de6bc8349a P d75e24e2145540bf893c6234d519d124: Deleting consensus metadata
I20260811 07:00:57.403339 5770 catalog_manager.cc:4841] TS d75e24e2145540bf893c6234d519d124 (127.4.40.2:44775): tablet fc44ea5447794a868cae74de6bc8349a (table test-table [id=f212e41d44164e27986fed567c46928b]) successfully deleted
I20260811 07:00:57.410089 6680 ts_tablet_manager.cc:1918] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:00:57.410512 6680 log.cc:1196] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/fc44ea5447794a868cae74de6bc8349a
I20260811 07:00:57.411365 6680 ts_tablet_manager.cc:1939] T fc44ea5447794a868cae74de6bc8349a P 7453613c1cac4b5d90a8edf94074ff07: Deleting consensus metadata
I20260811 07:00:57.413542 5772 catalog_manager.cc:4841] TS 7453613c1cac4b5d90a8edf94074ff07 (127.4.40.3:44067): tablet fc44ea5447794a868cae74de6bc8349a (table test-table [id=f212e41d44164e27986fed567c46928b]) successfully deleted
W20260811 07:00:57.414144 5772 catalog_manager.cc:4808] TS 7453613c1cac4b5d90a8edf94074ff07 (127.4.40.3:44067): delete failed for tablet fc44ea5447794a868cae74de6bc8349a because the tablet was not found. No further retry: Not found: Tablet not found: fc44ea5447794a868cae74de6bc8349a
I20260811 07:00:57.511603 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5900
I20260811 07:00:57.539661 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6135
W20260811 07:00:57.539695 5775 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:42117 (error 108)
I20260811 07:00:57.541831 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.541350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43844)
0811 07:00:57.541543 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.541670 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 07:00:57.542042 5775 catalog_manager.cc:4584] TS 29edc79c41d24dcbb1f93a95e3fa16ff (127.4.40.1:42117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc44ea5447794a868cae74de6bc8349a: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
I20260811 07:00:57.549955 6637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.541149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.4.40.2:44775)
0811 07:00:57.541326 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.541333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.541767 (+ 434us) server_negotiation.cc:407] Connection header received
0811 07:00:57.542491 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.542498 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.542618 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.542727 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:57.543224 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.544378 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.546337 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.546955 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.547081 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.548276 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:57.548287 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:57.548305 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:57.549081 (+ 776us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:57.549155 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:57.549180 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:57.549297 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.549472 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.549591 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 07:00:57.550629 6368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:57.595762 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.595184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43852)
0811 07:00:57.595437 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.595557 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 07:00:57.597049 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.596625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43864)
0811 07:00:57.596801 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.596884 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 07:00:57.642648 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6135
I20260811 07:00:57.657750 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.657209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43876)
0811 07:00:57.657419 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.657549 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 07:00:57.658885 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.658520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43884)
0811 07:00:57.658674 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.658768 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 07:00:57.679553 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6383
I20260811 07:00:57.687835 6636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.680387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59898 (local address 127.4.40.3:44067)
0811 07:00:57.680578 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 07:00:57.680582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:00:57.680697 (+ 115us) server_negotiation.cc:407] Connection header received
0811 07:00:57.680930 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:00:57.680934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:00:57.681016 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:00:57.681111 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:00:57.681649 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.682519 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.684420 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:00:57.684957 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:00:57.685056 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:00:57.686361 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:00:57.686368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:00:57.686383 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:00:57.687005 (+ 622us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:00:57.687049 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:00:57.687069 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:00:57.687157 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 07:00:57.687357 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 07:00:57.687436 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:00:57.688668 6602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:57.739602 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.738991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43894)
0811 07:00:57.739234 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.739356 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 07:00:57.740790 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.740349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43896)
0811 07:00:57.740572 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.740654 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 07:00:57.809036 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6383
I20260811 07:00:57.839784 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 5742
I20260811 07:00:57.841065 5848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:00:57.950918 6633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:00:57.950184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:42117 (local address 127.0.0.1:43898)
0811 07:00:57.950386 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 07:00:57.950578 (+ 192us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:42117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 07:00:57.951478 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 5742
2026-08-11T07:00:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2955 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:00:58 dist-test-slave-7vjv krb5kdc[6702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:00:58 dist-test-slave-7vjv krb5kdc[6702](info): set up 2 sockets
Aug 11 07:00:58 dist-test-slave-7vjv krb5kdc[6702](info): commencing operation
krb5kdc: starting...
W20260811 07:00:59.343015 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.340s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:00:59 dist-test-slave-7vjv krb5kdc[6702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:00:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:00:59.668408 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43153
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:43153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:00.005060 6724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:00.005340 6724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:00.005432 6724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:00.012379 6724 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:00.012478 6724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:00.012526 6724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:00.012590 6724 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:00.012635 6724 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:00.022109 6724 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:43153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:00.023308 6724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:00.024603 6724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:00.036190 6737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:00.036136 6739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:00.036234 6735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:00.041625 6724 server_base.cc:1034] running on GCE node
I20260811 07:01:00.042399 6724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:00.043721 6724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:00.044922 6724 hybrid_clock.cc:648] HybridClock initialized: now 1786431660044867 us; error 38 us; skew 500 ppm
I20260811 07:01:00.045291 6724 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:00 dist-test-slave-7vjv krb5kdc[6702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431660, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:00.048544 6724 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:00.050110 6724 webserver.cc:460] Webserver started at http://127.4.40.62:44413/ using document root <none> and password file <none>
I20260811 07:01:00.050830 6724 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:00.050984 6724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:00.051293 6724 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:00.054327 6724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "bf309bc76ce343ce895a998dc0496db1"
format_stamp: "Formatted at 2026-08-11 07:01:00 on dist-test-slave-7vjv"
server_key: "2e032959abe50aa05ecd5bc19e6b0cf3"
server_key_iv: "a5fa3df7cf4947a3386cc2ef8e4634c1"
server_key_version: "encryptionkey@0"
I20260811 07:01:00.055104 6724 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "bf309bc76ce343ce895a998dc0496db1"
format_stamp: "Formatted at 2026-08-11 07:01:00 on dist-test-slave-7vjv"
server_key: "2e032959abe50aa05ecd5bc19e6b0cf3"
server_key_iv: "a5fa3df7cf4947a3386cc2ef8e4634c1"
server_key_version: "encryptionkey@0"
I20260811 07:01:00.060267 6724 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 07:01:00.063620 6749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:00.064823 6724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:00.065057 6724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "bf309bc76ce343ce895a998dc0496db1"
format_stamp: "Formatted at 2026-08-11 07:01:00 on dist-test-slave-7vjv"
server_key: "2e032959abe50aa05ecd5bc19e6b0cf3"
server_key_iv: "a5fa3df7cf4947a3386cc2ef8e4634c1"
server_key_version: "encryptionkey@0"
I20260811 07:01:00.065249 6724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:00.085397 6724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:00.089788 6724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:00.090083 6724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:00.108798 6724 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:43153
I20260811 07:01:00.108837 6847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:43153 every 8 connection(s)
I20260811 07:01:00.110087 6724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:00.111883 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6724
I20260811 07:01:00.112136 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:00.112715 4256 external_mini_cluster.cc:1428] Setting key 0429037381cf208a74e771ebb44126d9
I20260811 07:01:00.114409 6849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:00.125329 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1: Bootstrap starting.
Aug 11 07:01:00 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:00.128958 6849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:00.130056 6849 log.cc:824] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:00.133291 6849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1: No bootstrap required, opened a new log
I20260811 07:01:00.135473 6855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:00.114667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43530 (local address 127.4.40.62:43153)
0811 07:01:00.115459 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:00.115471 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:00.115511 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:01:00.117236 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:00.117255 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:00.117614 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:00.118020 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:00.119607 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:00.120952 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:00.122274 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:00.122976 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:00.123194 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:00.127729 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:00.127736 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:00.127748 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:00.127784 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:00.131421 (+ 3637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:00.132258 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:00.132263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:00.132269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:00.132441 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:00.132918 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:00.132922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:00.132924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:00.133443 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:00.133667 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:00.134127 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 07:01:00.134436 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":375,"threads_started":1}
I20260811 07:01:00.141115 6849 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf309bc76ce343ce895a998dc0496db1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43153 } }
I20260811 07:01:00.141465 6849 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:00.141589 6849 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf309bc76ce343ce895a998dc0496db1, State: Initialized, Role: FOLLOWER
I20260811 07:01:00.142117 6849 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf309bc76ce343ce895a998dc0496db1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43153 } }
I20260811 07:01:00.142338 6849 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:00.142485 6849 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:00.142642 6849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:00.144619 6849 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf309bc76ce343ce895a998dc0496db1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43153 } }
I20260811 07:01:00.145107 6849 leader_election.cc:302] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf309bc76ce343ce895a998dc0496db1; no voters:
I20260811 07:01:00.145653 6849 leader_election.cc:288] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:00.145758 6858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:00.146533 6858 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [term 1 LEADER]: Becoming Leader. State: Replica: bf309bc76ce343ce895a998dc0496db1, State: Running, Role: LEADER
I20260811 07:01:00.147130 6858 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf309bc76ce343ce895a998dc0496db1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43153 } }
I20260811 07:01:00.147732 6849 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:00.150295 6859 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf309bc76ce343ce895a998dc0496db1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf309bc76ce343ce895a998dc0496db1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43153 } } }
I20260811 07:01:00.150593 6859 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:00.150988 6860 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf309bc76ce343ce895a998dc0496db1. Latest consensus state: current_term: 1 leader_uuid: "bf309bc76ce343ce895a998dc0496db1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf309bc76ce343ce895a998dc0496db1" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43153 } } }
I20260811 07:01:00.151373 6860 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:00.152709 6869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:00.157280 6869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:00.166170 6869 catalog_manager.cc:1347] Generated new cluster ID: 0010831c150e4a19a664776be97bf646
I20260811 07:01:00.166277 6869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:00.176751 6869 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:00.178422 6869 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:00.191998 6869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1: Generated new TSK 0
I20260811 07:01:00.193065 6869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.40.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
Entry for principal kudu/127.4.40.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab.
I20260811 07:01:00.279302 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43153
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:00.624212 6892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:00.624522 6892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:00.624706 6892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:00.631820 6892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:00.631940 6892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:00.632190 6892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:01:00.641733 6892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.1.keytab
--principal=kudu/127.4.40.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:00.642915 6892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:00.644296 6892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:00.657723 6906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:00.657766 6903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:00.657943 6902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:00.659255 6892 server_base.cc:1034] running on GCE node
I20260811 07:01:00.659890 6892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:00.660903 6892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:00.662154 6892 hybrid_clock.cc:648] HybridClock initialized: now 1786431660662125 us; error 53 us; skew 500 ppm
I20260811 07:01:00.662453 6892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:00 dist-test-slave-7vjv krb5kdc[6702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431660, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:00.665772 6892 init.cc:377] Logged in from keytab as kudu/127.4.40.1@KRBTEST.COM (short username kudu)
I20260811 07:01:00.667215 6892 webserver.cc:460] Webserver started at http://127.4.40.1:41027/ using document root <none> and password file <none>
I20260811 07:01:00.667888 6892 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:00.668046 6892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:00.668443 6892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:00.671403 6892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "da95425fabdb45e991970bf2d95928d3"
format_stamp: "Formatted at 2026-08-11 07:01:00 on dist-test-slave-7vjv"
server_key: "2047786861841a8bc168171390b17f11"
server_key_iv: "8c85610b2f8c1c581700d55098043cae"
server_key_version: "encryptionkey@0"
I20260811 07:01:00.672204 6892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "da95425fabdb45e991970bf2d95928d3"
format_stamp: "Formatted at 2026-08-11 07:01:00 on dist-test-slave-7vjv"
server_key: "2047786861841a8bc168171390b17f11"
server_key_iv: "8c85610b2f8c1c581700d55098043cae"
server_key_version: "encryptionkey@0"
I20260811 07:01:00.677196 6892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 07:01:00.680503 6921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:00.681806 6892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 07:01:00.681993 6892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "da95425fabdb45e991970bf2d95928d3"
format_stamp: "Formatted at 2026-08-11 07:01:00 on dist-test-slave-7vjv"
server_key: "2047786861841a8bc168171390b17f11"
server_key_iv: "8c85610b2f8c1c581700d55098043cae"
server_key_version: "encryptionkey@0"
I20260811 07:01:00.682204 6892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:00.700785 6892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:00.704784 6892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:00.705088 6892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:00.706030 6892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:01:00.707338 6892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:01:00.707440 6892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:00.707590 6892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:01:00.707662 6892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:00.741425 6892 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:34619
I20260811 07:01:00.741474 7118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:34619 every 8 connection(s)
I20260811 07:01:00.742751 6892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:01:00.752369 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 6892
I20260811 07:01:00.752588 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 07:01:00.753131 4256 external_mini_cluster.cc:1428] Setting key 0a6d52424bae30a1eb423d39ba9b553b
Aug 11 07:01:00 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431660, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.1@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:00.763640 7125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:00.745362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:57157 (local address 127.4.40.62:43153)
0811 07:01:00.745822 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:00.745829 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:00.746835 (+ 1006us) server_negotiation.cc:407] Connection header received
0811 07:01:00.748807 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:00.748811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:00.748876 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:00.749041 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:00.750454 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:00.751222 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:00.753262 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:00.753696 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:00.753844 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:00.758068 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:00.758073 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:00.758076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:00.758104 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:00.760687 (+ 2583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:00.761396 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:00.761404 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:00.761411 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:00.761526 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:00.761974 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:00.761980 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:00.761988 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:00.762224 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:00.762377 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:00.763026 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 07:01:00.763258 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":221,"threads_started":1}
I20260811 07:01:00.764649 7128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:00.745962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43153 (local address 127.4.40.1:57157)
0811 07:01:00.746678 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:00.746718 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:00.748538 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:00.749232 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:00.749238 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:00.749751 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:00.750308 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:00.750324 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:00.751360 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:00.751364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:00.753090 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:00.753106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:00.753971 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:00.753979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:00.754243 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:00.754957 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:00.754985 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:00.757814 (+ 2829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:00.760838 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:00.760847 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:00.760861 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:00.761274 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:00.761660 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:00.761665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:00.761668 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:00.761855 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:00.762353 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:00.762360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:00.762667 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 07:01:00.762912 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 07:01:00.763424 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":285,"threads_started":1}
I20260811 07:01:00.766109 7121 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43153
I20260811 07:01:00.766440 7121 heartbeater.cc:464] Registering TS with master...
I20260811 07:01:00.767290 7121 heartbeater.cc:511] Master 127.4.40.62:43153 requested a full tablet report, sending...
I20260811 07:01:00.769529 6781 ts_manager.cc:194] Registered new tserver with Master: da95425fabdb45e991970bf2d95928d3 (127.4.40.1:34619)
I20260811 07:01:00.771119 6781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.1@KRBTEST.COM'} at 127.4.40.1:57157
WARNING: no policy specified for kudu/127.4.40.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
Entry for principal kudu/127.4.40.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab.
I20260811 07:01:00.824821 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43153
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:01.178164 7132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:01.178437 7132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:01.178539 7132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:01.187229 7132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:01.187353 7132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:01.187486 7132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:01:01.197297 7132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.2.keytab
--principal=kudu/127.4.40.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:01.198585 7132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:01.199883 7132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:01.213177 7140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:01.213093 7152 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:01.213150 7141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:01.214468 7132 server_base.cc:1034] running on GCE node
I20260811 07:01:01.215144 7132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:01.216171 7132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:01.217362 7132 hybrid_clock.cc:648] HybridClock initialized: now 1786431661217294 us; error 60 us; skew 500 ppm
I20260811 07:01:01.217706 7132 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:01 dist-test-slave-7vjv krb5kdc[6702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431661, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:01.220970 7132 init.cc:377] Logged in from keytab as kudu/127.4.40.2@KRBTEST.COM (short username kudu)
I20260811 07:01:01.222553 7132 webserver.cc:460] Webserver started at http://127.4.40.2:34293/ using document root <none> and password file <none>
I20260811 07:01:01.223284 7132 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:01.223443 7132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:01.223785 7132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:01.226838 7132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "82663ba91a344719807b493db00587d4"
format_stamp: "Formatted at 2026-08-11 07:01:01 on dist-test-slave-7vjv"
server_key: "8689835e61dc5bab2d208824664035d8"
server_key_iv: "a94d643ff7b64db478196f404caaa856"
server_key_version: "encryptionkey@0"
I20260811 07:01:01.227681 7132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "82663ba91a344719807b493db00587d4"
format_stamp: "Formatted at 2026-08-11 07:01:01 on dist-test-slave-7vjv"
server_key: "8689835e61dc5bab2d208824664035d8"
server_key_iv: "a94d643ff7b64db478196f404caaa856"
server_key_version: "encryptionkey@0"
I20260811 07:01:01.233582 7132 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 07:01:01.237164 7170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:01.238436 7132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:01:01.238631 7132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "82663ba91a344719807b493db00587d4"
format_stamp: "Formatted at 2026-08-11 07:01:01 on dist-test-slave-7vjv"
server_key: "8689835e61dc5bab2d208824664035d8"
server_key_iv: "a94d643ff7b64db478196f404caaa856"
server_key_version: "encryptionkey@0"
I20260811 07:01:01.238837 7132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:01.260156 7132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:01.263863 7132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:01.264237 7132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:01.265198 7132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:01:01.266611 7132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:01:01.266731 7132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:01.266824 7132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:01:01.266865 7132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:01.301012 7132 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:42397
I20260811 07:01:01.301060 7365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:42397 every 8 connection(s)
I20260811 07:01:01.302318 7132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:01:01.307641 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7132
I20260811 07:01:01.307861 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 07:01:01.308439 4256 external_mini_cluster.cc:1428] Setting key aca3a9744bf67181070aa20e4c6a1ff2
Aug 11 07:01:01 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431661, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.2@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:01.322494 7370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.304892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:51423 (local address 127.4.40.62:43153)
0811 07:01:01.305382 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:01.305386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:01.306162 (+ 776us) server_negotiation.cc:407] Connection header received
0811 07:01:01.308408 (+ 2246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:01.308413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:01.308536 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:01.308670 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:01.310167 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.310910 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.312533 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.312979 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.313123 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.316622 (+ 3499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:01.316627 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:01.316630 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:01.316660 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:01.319361 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:01.320179 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:01.320184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:01.320189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:01.320325 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:01.320772 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:01.320776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:01.320778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:01.321036 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:01.321195 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:01.321878 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 07:01:01.322102 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":284,"threads_started":1}
I20260811 07:01:01.323431 7371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.305311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43153 (local address 127.4.40.2:51423)
0811 07:01:01.306002 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:01.306040 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:01.308069 (+ 2029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:01.308916 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:01.308924 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:01.309435 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:01.309989 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.310006 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.311068 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.311074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.312373 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.312386 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.313215 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.313223 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.313455 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.314140 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:01.314167 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:01.316404 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:01.319547 (+ 3143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:01.319555 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:01.319570 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:01.320024 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:01.320443 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:01.320448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:01.320451 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:01.320644 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:01.321169 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:01.321177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:01.321499 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 07:01:01.321766 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 07:01:01.322289 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":273,"threads_started":1}
I20260811 07:01:01.324985 7367 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43153
I20260811 07:01:01.325307 7367 heartbeater.cc:464] Registering TS with master...
I20260811 07:01:01.326006 7367 heartbeater.cc:511] Master 127.4.40.62:43153 requested a full tablet report, sending...
I20260811 07:01:01.327443 6781 ts_manager.cc:194] Registered new tserver with Master: 82663ba91a344719807b493db00587d4 (127.4.40.2:42397)
I20260811 07:01:01.328315 6781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.2@KRBTEST.COM'} at 127.4.40.2:51423
WARNING: no policy specified for kudu/127.4.40.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
Entry for principal kudu/127.4.40.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab.
I20260811 07:01:01.382624 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43153
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:01.725402 7375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:01.725669 7375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:01.725764 7375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:01.732733 7375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:01.732838 7375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:01.732963 7375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:01:01.743913 7375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.3.keytab
--principal=kudu/127.4.40.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:43153
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:01.745460 7375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:01.747184 7375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:01.760522 7382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:01.760821 7383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:01.761108 7385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:01.762194 7375 server_base.cc:1034] running on GCE node
I20260811 07:01:01.762862 7375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:01.763836 7375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:01.765074 7375 hybrid_clock.cc:648] HybridClock initialized: now 1786431661765032 us; error 35 us; skew 500 ppm
I20260811 07:01:01.765417 7375 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:01 dist-test-slave-7vjv krb5kdc[6702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431661, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:01.769024 7375 init.cc:377] Logged in from keytab as kudu/127.4.40.3@KRBTEST.COM (short username kudu)
I20260811 07:01:01.770541 7375 webserver.cc:460] Webserver started at http://127.4.40.3:38015/ using document root <none> and password file <none>
I20260811 07:01:01.771236 7375 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:01.771384 7375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:01.771699 7375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:01.773792 7121 heartbeater.cc:503] Master 127.4.40.62:43153 was elected leader, sending a full tablet report...
I20260811 07:01:01.775080 7375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "fa3abbe7ead2465f9f80e64226e46137"
format_stamp: "Formatted at 2026-08-11 07:01:01 on dist-test-slave-7vjv"
server_key: "86dd651c0f718a4c3bb81cd7e8a56efc"
server_key_iv: "1e9c59b57697d91745b60062e7e3ed31"
server_key_version: "encryptionkey@0"
I20260811 07:01:01.776178 7375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "fa3abbe7ead2465f9f80e64226e46137"
format_stamp: "Formatted at 2026-08-11 07:01:01 on dist-test-slave-7vjv"
server_key: "86dd651c0f718a4c3bb81cd7e8a56efc"
server_key_iv: "1e9c59b57697d91745b60062e7e3ed31"
server_key_version: "encryptionkey@0"
I20260811 07:01:01.781209 7375 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 07:01:01.784493 7394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:01.785815 7375 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 07:01:01.786021 7375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "fa3abbe7ead2465f9f80e64226e46137"
format_stamp: "Formatted at 2026-08-11 07:01:01 on dist-test-slave-7vjv"
server_key: "86dd651c0f718a4c3bb81cd7e8a56efc"
server_key_iv: "1e9c59b57697d91745b60062e7e3ed31"
server_key_version: "encryptionkey@0"
I20260811 07:01:01.786244 7375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:01.801658 7375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:01.805614 7375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:01.805915 7375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:01.806849 7375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:01:01.808151 7375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:01:01.808254 7375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:01.808404 7375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:01:01.808490 7375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:01.842674 7375 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:36459
I20260811 07:01:01.842729 7610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:36459 every 8 connection(s)
I20260811 07:01:01.844039 7375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:01:01.845610 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7375
I20260811 07:01:01.845834 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 07:01:01.846433 4256 external_mini_cluster.cc:1428] Setting key acf74f36255ba066119236fdc28f44d6
Aug 11 07:01:01 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431661, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.3@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:01.864880 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.847176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:38377 (local address 127.4.40.62:43153)
0811 07:01:01.847745 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:01.847750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:01.848945 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 07:01:01.851024 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:01.851028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:01.851094 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:01.851184 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:01.852723 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.853457 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.854888 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.855289 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.855383 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.858795 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:01.858800 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:01.858803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:01.858839 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:01.861783 (+ 2944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:01.862559 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:01.862564 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:01.862568 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:01.862708 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:01.863156 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:01.863161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:01.863166 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:01.863422 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:01.863590 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:01.864272 (+ 682us) server_negotiation.cc:299] Negotiation successful
0811 07:01:01.864493 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":317,"threads_started":1}
I20260811 07:01:01.865749 7616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.847770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43153 (local address 127.4.40.3:38377)
0811 07:01:01.848777 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:01.848837 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:01.850771 (+ 1934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:01.851396 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:01.851404 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:01.851929 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:01.852585 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.852603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.853597 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.853600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.854766 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.854778 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.855482 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.855486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.855671 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.856346 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:01.856379 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:01.858587 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:01.861911 (+ 3324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:01.861919 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:01.861933 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:01.862421 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:01.862824 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:01.862829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:01.862832 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:01.863027 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:01.863555 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:01.863564 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:01.863888 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 07:01:01.864175 (+ 287us) client_negotiation.cc:240] Negotiation successful
0811 07:01:01.864667 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":407,"threads_started":1}
I20260811 07:01:01.867049 7612 heartbeater.cc:347] Connected to a master server at 127.4.40.62:43153
I20260811 07:01:01.867305 7612 heartbeater.cc:464] Registering TS with master...
I20260811 07:01:01.867967 7612 heartbeater.cc:511] Master 127.4.40.62:43153 requested a full tablet report, sending...
I20260811 07:01:01.869486 6781 ts_manager.cc:194] Registered new tserver with Master: fa3abbe7ead2465f9f80e64226e46137 (127.4.40.3:36459)
I20260811 07:01:01.870333 6781 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.40.3@KRBTEST.COM'} at 127.4.40.3:38377
I20260811 07:01:01.874262 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:01:01.890712 7615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.878488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43544 (local address 127.4.40.62:43153)
0811 07:01:01.878653 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:01.878658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:01.879100 (+ 442us) server_negotiation.cc:407] Connection header received
0811 07:01:01.879226 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:01.879230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:01.879284 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:01.879363 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:01.880897 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.881619 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.882850 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.883270 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.883359 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.885733 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:01.885737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:01.885740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:01.885769 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:01.888306 (+ 2537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:01.888892 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:01.888895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:01.888898 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:01.889073 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:01.889413 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:01.889416 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:01.889418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:01.889734 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:01.889944 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:01.890285 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 07:01:01.890491 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:01:01.894428 6781 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43544:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:01:01.901376 6781 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:01:01.925231 7635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.914566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:36459 (local address 127.0.0.1:43134)
0811 07:01:01.915148 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:01.915166 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:01.915309 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:01.917370 (+ 2061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:01.917377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:01.917407 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:01.917986 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.918012 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.920356 (+ 2344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.920361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.922349 (+ 1988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.922364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.924154 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.924161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.924335 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.924376 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 07:01:01.924536 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 07:01:01.924733 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"mutex_wait_us":78,"spinlock_wait_cycles":768,"thread_start_us":739,"threads_started":1}
I20260811 07:01:01.925483 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.913520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44716 (local address 127.4.40.1:34619)
0811 07:01:01.917085 (+ 3565us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:01.917093 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:01.917123 (+ 30us) server_negotiation.cc:407] Connection header received
0811 07:01:01.917221 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:01.917229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:01.917481 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:01.917681 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:01.918588 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.919630 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.922118 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.923524 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.923663 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.923916 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:01.924936 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0811 07:01:01.925136 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2418,"thread_start_us":956,"threads_started":1}
I20260811 07:01:01.925904 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.915125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.4.40.3:36459)
0811 07:01:01.916708 (+ 1583us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:01.916719 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:01.916756 (+ 37us) server_negotiation.cc:407] Connection header received
0811 07:01:01.916859 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:01.916868 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:01.917146 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:01.917353 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:01.918251 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.920196 (+ 1945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.922534 (+ 2338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.923914 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.924057 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.924301 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:01.925147 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 07:01:01.925333 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1412,"thread_start_us":1234,"threads_started":1}
I20260811 07:01:01.926028 7633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.913174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34619 (local address 127.0.0.1:44716)
0811 07:01:01.913971 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:01.914015 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:01.914181 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:01.917669 (+ 3488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:01.917676 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:01.917696 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:01.918393 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.918404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.919774 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.919782 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.921949 (+ 2167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.921962 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.923736 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.923742 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.923923 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.924496 (+ 573us) client_negotiation.cc:769] Sending connection context
0811 07:01:01.924594 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 07:01:01.924759 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":215,"thread_start_us":365,"threads_started":1}
I20260811 07:01:01.926090 7639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.914739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:42397 (local address 127.0.0.1:47296)
0811 07:01:01.915899 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:01.915922 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:01.916065 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:01.917559 (+ 1494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:01.917564 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:01.917609 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:01.918376 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.918389 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.920356 (+ 1967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.920361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.923369 (+ 3008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:01.923381 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.925254 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.925263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:01.925422 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.925462 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 07:01:01.925535 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 07:01:01.925691 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":138,"spinlock_wait_cycles":1792,"thread_start_us":841,"threads_started":1}
I20260811 07:01:01.927206 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:01.915035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47296 (local address 127.4.40.2:42397)
0811 07:01:01.916985 (+ 1950us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:01.916994 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:01.917018 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:01:01.917105 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:01.917113 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:01.917325 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:01.917976 (+ 651us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:01.918610 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.920193 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.923544 (+ 3351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:01.925004 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:01.926403 (+ 1399us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:01.926621 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:01.926687 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 07:01:01.926839 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1787,"thread_start_us":1674,"threads_started":1}
I20260811 07:01:01.930049 7253 tablet_service.cc:1467] Processing CreateTablet for tablet 0b210e15dd7e4e778e493fba2ae2cf57 (DEFAULT_TABLE table=test-table [id=d8cf48e94619428086993343a3ae3c46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:01:01.930018 7004 tablet_service.cc:1467] Processing CreateTablet for tablet 0b210e15dd7e4e778e493fba2ae2cf57 (DEFAULT_TABLE table=test-table [id=d8cf48e94619428086993343a3ae3c46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:01:01.930008 7488 tablet_service.cc:1467] Processing CreateTablet for tablet 0b210e15dd7e4e778e493fba2ae2cf57 (DEFAULT_TABLE table=test-table [id=d8cf48e94619428086993343a3ae3c46]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:01:01.931414 7253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b210e15dd7e4e778e493fba2ae2cf57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:01.931427 7488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b210e15dd7e4e778e493fba2ae2cf57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:01.931741 7004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b210e15dd7e4e778e493fba2ae2cf57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:01.942529 7646 tablet_bootstrap.cc:492] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Bootstrap starting.
I20260811 07:01:01.946324 7647 tablet_bootstrap.cc:492] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Bootstrap starting.
I20260811 07:01:01.946895 7646 tablet_bootstrap.cc:654] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:01.948279 7646 log.cc:824] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:01.949749 7648 tablet_bootstrap.cc:492] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Bootstrap starting.
I20260811 07:01:01.950086 7647 tablet_bootstrap.cc:654] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:01.951319 7646 tablet_bootstrap.cc:492] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: No bootstrap required, opened a new log
I20260811 07:01:01.951346 7647 log.cc:824] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:01.951618 7646 ts_tablet_manager.cc:1397] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 07:01:01.953670 7648 tablet_bootstrap.cc:654] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:01.954854 7647 tablet_bootstrap.cc:492] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: No bootstrap required, opened a new log
I20260811 07:01:01.954918 7648 log.cc:824] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:01.955114 7647 ts_tablet_manager.cc:1397] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 07:01:01.957710 7648 tablet_bootstrap.cc:492] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: No bootstrap required, opened a new log
I20260811 07:01:01.957954 7648 ts_tablet_manager.cc:1397] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 07:01:01.962414 7646 raft_consensus.cc:348] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:01.962874 7646 raft_consensus.cc:374] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:01.962980 7646 raft_consensus.cc:729] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82663ba91a344719807b493db00587d4, State: Initialized, Role: FOLLOWER
I20260811 07:01:01.963783 7646 consensus_queue.cc:260] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:01.965051 7367 heartbeater.cc:503] Master 127.4.40.62:43153 was elected leader, sending a full tablet report...
I20260811 07:01:01.965762 7646 ts_tablet_manager.cc:1428] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 07:01:01.966162 7647 raft_consensus.cc:348] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:01.966609 7647 raft_consensus.cc:374] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:01.966761 7647 raft_consensus.cc:729] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3abbe7ead2465f9f80e64226e46137, State: Initialized, Role: FOLLOWER
I20260811 07:01:01.967144 7648 raft_consensus.cc:348] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:01.967370 7647 consensus_queue.cc:260] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:01.967572 7648 raft_consensus.cc:374] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:01.967715 7648 raft_consensus.cc:729] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da95425fabdb45e991970bf2d95928d3, State: Initialized, Role: FOLLOWER
I20260811 07:01:01.968595 7612 heartbeater.cc:503] Master 127.4.40.62:43153 was elected leader, sending a full tablet report...
I20260811 07:01:01.968422 7648 consensus_queue.cc:260] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:01.969287 7647 ts_tablet_manager.cc:1428] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260811 07:01:01.971136 7648 ts_tablet_manager.cc:1428] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Time spent starting tablet: real 0.013s user 0.001s sys 0.009s
W20260811 07:01:01.995270 7122 tablet.cc:2369] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:01:02.054522 7368 tablet.cc:2369] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 07:01:02.096882 7613 tablet.cc:2369] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:01:02.213758 7656 raft_consensus.cc:482] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:01:02.214038 7656 raft_consensus.cc:504] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:02.215647 7656 leader_election.cc:288] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da95425fabdb45e991970bf2d95928d3 (127.4.40.1:34619), fa3abbe7ead2465f9f80e64226e46137 (127.4.40.3:36459)
I20260811 07:01:02.224462 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.217246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:58189 (local address 127.4.40.3:36459)
0811 07:01:02.217453 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:02.217459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:02.217687 (+ 228us) server_negotiation.cc:407] Connection header received
0811 07:01:02.217845 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:02.217849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:02.217906 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:02.217979 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:02.218485 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.219478 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.221844 (+ 2366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.223163 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.223294 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.223400 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:02.224051 (+ 651us) server_negotiation.cc:299] Negotiation successful
0811 07:01:02.224204 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 07:01:02.224462 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.216186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:49071 (local address 127.4.40.1:34619)
0811 07:01:02.216469 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:02.216478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:02.216919 (+ 441us) server_negotiation.cc:407] Connection header received
0811 07:01:02.217202 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:02.217211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:02.217310 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:02.217422 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:02.218101 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.219171 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.221818 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.223150 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.223295 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.223399 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:02.224051 (+ 652us) server_negotiation.cc:299] Negotiation successful
0811 07:01:02.224204 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 07:01:02.224752 7659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.216051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:34619 (local address 127.4.40.2:49071)
0811 07:01:02.216808 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:02.216824 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:02.216999 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:02.217447 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:02.217451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:02.217481 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:02.217915 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:02.217925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.219301 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.219304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:02.221636 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:02.221657 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.223448 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.223454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:02.223756 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.223894 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 07:01:02.223996 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 07:01:02.224229 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":651,"mutex_wait_us":160,"thread_start_us":477,"threads_started":1}
I20260811 07:01:02.224774 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.216946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:36459 (local address 127.4.40.2:58189)
0811 07:01:02.217572 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:02.217595 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:02.217727 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:02.218025 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:02.218029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:02.218058 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:02.218343 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:02.218351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.219622 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.219628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:02.221709 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:02.221723 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.223453 (+ 1730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.223457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:02.223756 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.223894 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 07:01:02.223996 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 07:01:02.224229 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":158,"thread_start_us":508,"threads_started":1}
I20260811 07:01:02.225932 7035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b210e15dd7e4e778e493fba2ae2cf57" candidate_uuid: "82663ba91a344719807b493db00587d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da95425fabdb45e991970bf2d95928d3" is_pre_election: true
I20260811 07:01:02.225937 7530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b210e15dd7e4e778e493fba2ae2cf57" candidate_uuid: "82663ba91a344719807b493db00587d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3abbe7ead2465f9f80e64226e46137" is_pre_election: true
I20260811 07:01:02.226450 7035 raft_consensus.cc:2454] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82663ba91a344719807b493db00587d4 in term 0.
I20260811 07:01:02.226455 7530 raft_consensus.cc:2454] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82663ba91a344719807b493db00587d4 in term 0.
I20260811 07:01:02.227118 7174 leader_election.cc:302] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82663ba91a344719807b493db00587d4, da95425fabdb45e991970bf2d95928d3; no voters:
I20260811 07:01:02.227618 7656 raft_consensus.cc:2781] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:01:02.227780 7656 raft_consensus.cc:482] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:01:02.227908 7656 raft_consensus.cc:3037] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:02.230125 7656 raft_consensus.cc:504] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:02.230692 7656 leader_election.cc:288] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [CANDIDATE]: Term 1 election: Requested vote from peers da95425fabdb45e991970bf2d95928d3 (127.4.40.1:34619), fa3abbe7ead2465f9f80e64226e46137 (127.4.40.3:36459)
I20260811 07:01:02.231143 7035 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b210e15dd7e4e778e493fba2ae2cf57" candidate_uuid: "82663ba91a344719807b493db00587d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da95425fabdb45e991970bf2d95928d3"
I20260811 07:01:02.231189 7530 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b210e15dd7e4e778e493fba2ae2cf57" candidate_uuid: "82663ba91a344719807b493db00587d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa3abbe7ead2465f9f80e64226e46137"
I20260811 07:01:02.231333 7035 raft_consensus.cc:3037] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:02.231423 7530 raft_consensus.cc:3037] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:02.233861 7530 raft_consensus.cc:2454] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82663ba91a344719807b493db00587d4 in term 1.
I20260811 07:01:02.233928 7035 raft_consensus.cc:2454] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82663ba91a344719807b493db00587d4 in term 1.
I20260811 07:01:02.234297 7174 leader_election.cc:302] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82663ba91a344719807b493db00587d4, fa3abbe7ead2465f9f80e64226e46137; no voters:
I20260811 07:01:02.234589 7656 raft_consensus.cc:2781] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:02.235077 7656 raft_consensus.cc:686] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 1 LEADER]: Becoming Leader. State: Replica: 82663ba91a344719807b493db00587d4, State: Running, Role: LEADER
I20260811 07:01:02.235646 7656 consensus_queue.cc:237] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } }
I20260811 07:01:02.240324 6780 catalog_manager.cc:5495] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 82663ba91a344719807b493db00587d4 (127.4.40.2). New cstate: current_term: 1 leader_uuid: "82663ba91a344719807b493db00587d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82663ba91a344719807b493db00587d4" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 42397 } health_report { overall_health: HEALTHY } } }
I20260811 07:01:02.290012 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.284664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47302 (local address 127.4.40.2:42397)
0811 07:01:02.284890 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:02.284914 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:02.284934 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:01:02.285068 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:02.285072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:02.285162 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:02.285320 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:02.285722 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.286444 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.287767 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.288224 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.288349 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.288836 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:02.289374 (+ 538us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:02.289551 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:02.289641 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 07:01:02.289723 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 07:01:02.298533 7035 raft_consensus.cc:1261] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 1 FOLLOWER]: Refusing update from remote peer 82663ba91a344719807b493db00587d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:02.298524 7530 raft_consensus.cc:1261] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 1 FOLLOWER]: Refusing update from remote peer 82663ba91a344719807b493db00587d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:02.299541 7665 consensus_queue.cc:1035] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa3abbe7ead2465f9f80e64226e46137" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 36459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:02.299932 7656 consensus_queue.cc:1035] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da95425fabdb45e991970bf2d95928d3" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 34619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:02.312361 7672 mvcc.cc:205] Tried to move back new op lower bound from 7317224088763707392 to 7317224088521682944. Current Snapshot: MvccSnapshot[applied={T|T < 7317224088763707392}]
I20260811 07:01:02.325536 7674 mvcc.cc:205] Tried to move back new op lower bound from 7317224088763707392 to 7317224088521682944. Current Snapshot: MvccSnapshot[applied={T|T < 7317224088763707392}]
I20260811 07:01:02.334422 6777 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:01:02.334753 6777 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43544:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:01:02.340117 6777 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf309bc76ce343ce895a998dc0496db1: Sending DeleteTablet for 3 replicas of tablet 0b210e15dd7e4e778e493fba2ae2cf57
I20260811 07:01:02.341632 7488 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b210e15dd7e4e778e493fba2ae2cf57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:01:02 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:43134
I20260811 07:01:02.342725 7689 tablet_replica.cc:330] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: stopping tablet replica
I20260811 07:01:02.343386 7689 raft_consensus.cc:2229] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:01:02.343997 7689 raft_consensus.cc:2258] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:01:02.345082 7004 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b210e15dd7e4e778e493fba2ae2cf57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:01:02 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:44716
I20260811 07:01:02.345716 7253 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b210e15dd7e4e778e493fba2ae2cf57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:01:02 UTC) from {username='kudu', principal='kudu/127.4.40.62@KRBTEST.COM'} at 127.0.0.1:47296
I20260811 07:01:02.346063 7691 tablet_replica.cc:330] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: stopping tablet replica
I20260811 07:01:02.346532 7691 raft_consensus.cc:2229] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:01:02.346602 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6892
I20260811 07:01:02.347090 7691 raft_consensus.cc:2258] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:01:02.350778 7689 ts_tablet_manager.cc:1905] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:01:02.353717 7691 ts_tablet_manager.cc:1905] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:01:02.357045 7692 tablet_replica.cc:330] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: stopping tablet replica
I20260811 07:01:02.357643 7692 raft_consensus.cc:2229] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:01:02.358012 7689 ts_tablet_manager.cc:1918] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:01:02.358196 7689 log.cc:1196] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/0b210e15dd7e4e778e493fba2ae2cf57
I20260811 07:01:02.358723 7689 ts_tablet_manager.cc:1939] T 0b210e15dd7e4e778e493fba2ae2cf57 P fa3abbe7ead2465f9f80e64226e46137: Deleting consensus metadata
I20260811 07:01:02.358657 7692 raft_consensus.cc:2258] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:01:02.360597 6756 catalog_manager.cc:4841] TS fa3abbe7ead2465f9f80e64226e46137 (127.4.40.3:36459): tablet 0b210e15dd7e4e778e493fba2ae2cf57 (table test-table [id=d8cf48e94619428086993343a3ae3c46]) successfully deleted
I20260811 07:01:02.361253 7692 ts_tablet_manager.cc:1905] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:01:02.366781 7691 ts_tablet_manager.cc:1918] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:01:02.366982 7691 log.cc:1196] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/0b210e15dd7e4e778e493fba2ae2cf57
I20260811 07:01:02.367655 7691 ts_tablet_manager.cc:1939] T 0b210e15dd7e4e778e493fba2ae2cf57 P da95425fabdb45e991970bf2d95928d3: Deleting consensus metadata
I20260811 07:01:02.369223 7692 ts_tablet_manager.cc:1918] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:01:02.369452 7692 log.cc:1196] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/0b210e15dd7e4e778e493fba2ae2cf57
I20260811 07:01:02.369778 6756 catalog_manager.cc:4841] TS da95425fabdb45e991970bf2d95928d3 (127.4.40.1:34619): tablet 0b210e15dd7e4e778e493fba2ae2cf57 (table test-table [id=d8cf48e94619428086993343a3ae3c46]) successfully deleted
I20260811 07:01:02.370165 7692 ts_tablet_manager.cc:1939] T 0b210e15dd7e4e778e493fba2ae2cf57 P 82663ba91a344719807b493db00587d4: Deleting consensus metadata
I20260811 07:01:02.371531 6762 catalog_manager.cc:4841] TS 82663ba91a344719807b493db00587d4 (127.4.40.2:42397): tablet 0b210e15dd7e4e778e493fba2ae2cf57 (table test-table [id=d8cf48e94619428086993343a3ae3c46]) successfully deleted
Aug 11 07:01:02 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.1@KRBTEST.COM
I20260811 07:01:02.381649 7634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.348375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.4.40.1:34619)
0811 07:01:02.348565 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:02.348571 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:02.360966 (+ 12395us) server_negotiation.cc:407] Connection header received
0811 07:01:02.361465 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:02.361471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:02.361580 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:02.361657 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:02.363596 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.364871 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.366815 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.367528 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.367743 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.375649 (+ 7906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:02.375661 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:02.375674 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:02.375715 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:02.378557 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:02.379275 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:02.379291 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:02.379293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:02.379428 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:02.379849 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:02.379854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:02.379859 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:02.380308 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:02.380624 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:02.380957 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 07:01:02.381206 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:01:02.382407 7111 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:02.475163 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6892
I20260811 07:01:02.508508 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7132
Aug 11 07:01:02 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.2@KRBTEST.COM
I20260811 07:01:02.526304 7641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.509328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.4.40.2:42397)
0811 07:01:02.509573 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:02.509579 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:02.509602 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:01:02.509710 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:02.509715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:02.509807 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:02.509922 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:02.511194 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.512495 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.513755 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.514491 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.514626 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.518938 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:02.518961 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:02.518984 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:02.519034 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:02.523248 (+ 4214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:02.524020 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:02.524031 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:02.524040 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:02.524288 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:02.524682 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:02.524687 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:02.524691 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:02.525123 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:02.525511 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:02.525681 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 07:01:02.525908 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 07:01:02.527076 7360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:02.642503 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7132
I20260811 07:01:02.670281 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7375
Aug 11 07:01:02 dist-test-slave-7vjv krb5kdc[6702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431659, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.3@KRBTEST.COM
I20260811 07:01:02.685256 7645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:02.671072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.4.40.3:36459)
0811 07:01:02.671245 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:02.671249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:02.671330 (+ 81us) server_negotiation.cc:407] Connection header received
0811 07:01:02.671485 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:02.671489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:02.671566 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:02.671654 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:02.673091 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.673827 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.675100 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:02.675486 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:02.675576 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:02.680017 (+ 4441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:02.680031 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:02.680048 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:02.680081 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:02.682851 (+ 2770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:02.683460 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:02.683475 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:02.683477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:02.683605 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:02.683909 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:02.683913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:02.683915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:02.684344 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:02.684586 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:02.684822 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 07:01:02.684990 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:01:02.685904 7603 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:02.774850 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7375
I20260811 07:01:02.804033 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 6724
I20260811 07:01:02.805186 6841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:02.916443 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 6724
2026-08-11T07:01:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4964 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T07:01:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:02Z Disabled control of system clock
I20260811 07:01:03.020326 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45985
--webserver_interface=127.4.40.62
--webserver_port=0
--builtin_ntp_servers=127.4.40.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45985
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:01:03.423959 7704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:03.424346 7704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:03.424533 7704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:03.431967 7704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:03.432078 7704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:03.432202 7704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:03.432315 7704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:03.432422 7704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:03.441883 7704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45985
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:03.443199 7704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:03.445065 7704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:03.457396 7720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:03.457504 7716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:03.457518 7715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:03.458674 7704 server_base.cc:1034] running on GCE node
I20260811 07:01:03.459465 7704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:03.461144 7704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:03.462386 7704 hybrid_clock.cc:648] HybridClock initialized: now 1786431663462339 us; error 59 us; skew 500 ppm
I20260811 07:01:03.462740 7704 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:01:03.465049 7704 webserver.cc:460] Webserver started at http://127.4.40.62:37183/ using document root <none> and password file <none>
I20260811 07:01:03.465734 7704 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:03.465868 7704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:03.466194 7704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:03.469146 7704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "3582db3e119644e695fe6f27ce3a3ed8"
format_stamp: "Formatted at 2026-08-11 07:01:03 on dist-test-slave-7vjv"
server_key: "0afb52ea35f1e1b17827a68ce772319b"
server_key_iv: "ad568e074be5bb92f1002237d77f29e0"
server_key_version: "encryptionkey@0"
I20260811 07:01:03.470070 7704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "3582db3e119644e695fe6f27ce3a3ed8"
format_stamp: "Formatted at 2026-08-11 07:01:03 on dist-test-slave-7vjv"
server_key: "0afb52ea35f1e1b17827a68ce772319b"
server_key_iv: "ad568e074be5bb92f1002237d77f29e0"
server_key_version: "encryptionkey@0"
I20260811 07:01:03.475206 7704 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 07:01:03.478699 7729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:03.479988 7704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:01:03.480219 7704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "3582db3e119644e695fe6f27ce3a3ed8"
format_stamp: "Formatted at 2026-08-11 07:01:03 on dist-test-slave-7vjv"
server_key: "0afb52ea35f1e1b17827a68ce772319b"
server_key_iv: "ad568e074be5bb92f1002237d77f29e0"
server_key_version: "encryptionkey@0"
I20260811 07:01:03.480435 7704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:03.519134 7704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:03.520123 7704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:03.520448 7704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:03.539536 7704 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45985
I20260811 07:01:03.539559 7833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45985 every 8 connection(s)
I20260811 07:01:03.541070 7704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:03.544937 7835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:03.551065 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7704
I20260811 07:01:03.551281 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:03.551723 4256 external_mini_cluster.cc:1428] Setting key 20d178c01fdbcb9b520d8ca6cd581bb1
I20260811 07:01:03.556339 7835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8: Bootstrap starting.
I20260811 07:01:03.560482 7835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:03.561719 7835 log.cc:824] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:03.564918 7835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8: No bootstrap required, opened a new log
I20260811 07:01:03.567444 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:03.553848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54578 (local address 127.4.40.62:45985)
0811 07:01:03.554905 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:03.554920 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:03.554973 (+ 53us) server_negotiation.cc:407] Connection header received
0811 07:01:03.557412 (+ 2439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:03.557439 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:03.558053 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:03.558511 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:03.559073 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:03.561130 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:03.562857 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:03.563818 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:03.564003 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:03.565247 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:03.565254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:03.565266 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:03.565809 (+ 543us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:03.565880 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:03.565911 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:03.566044 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:03.566254 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 07:01:03.566541 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":303,"threads_started":1}
I20260811 07:01:03.572623 7835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3582db3e119644e695fe6f27ce3a3ed8" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45985 } }
I20260811 07:01:03.572966 7835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:03.573066 7835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3582db3e119644e695fe6f27ce3a3ed8, State: Initialized, Role: FOLLOWER
I20260811 07:01:03.573592 7835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3582db3e119644e695fe6f27ce3a3ed8" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45985 } }
I20260811 07:01:03.573833 7835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:03.573993 7835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:03.574183 7835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:03.576669 7835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3582db3e119644e695fe6f27ce3a3ed8" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45985 } }
I20260811 07:01:03.577159 7835 leader_election.cc:302] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3582db3e119644e695fe6f27ce3a3ed8; no voters:
I20260811 07:01:03.577730 7835 leader_election.cc:288] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:03.577936 7845 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:03.578877 7845 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [term 1 LEADER]: Becoming Leader. State: Replica: 3582db3e119644e695fe6f27ce3a3ed8, State: Running, Role: LEADER
I20260811 07:01:03.579456 7845 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3582db3e119644e695fe6f27ce3a3ed8" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45985 } }
I20260811 07:01:03.579877 7835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:03.582993 7846 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3582db3e119644e695fe6f27ce3a3ed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3582db3e119644e695fe6f27ce3a3ed8" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45985 } } }
I20260811 07:01:03.582886 7847 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3582db3e119644e695fe6f27ce3a3ed8. Latest consensus state: current_term: 1 leader_uuid: "3582db3e119644e695fe6f27ce3a3ed8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3582db3e119644e695fe6f27ce3a3ed8" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45985 } } }
I20260811 07:01:03.583393 7846 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:03.583428 7847 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:03.587621 7856 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:03.592891 7856 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:03.602993 7856 catalog_manager.cc:1347] Generated new cluster ID: 9ce442d71a6d44b1b04964310a94cccc
I20260811 07:01:03.603179 7856 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:03.639328 7856 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:03.641197 7856 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:03.648603 7856 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8: Generated new TSK 0
I20260811 07:01:03.649381 7856 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:03.657176 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.1:0
--local_ip_for_outbound_sockets=127.4.40.1
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45985
--builtin_ntp_servers=127.4.40.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 07:01:04.034550 7873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:04.034871 7873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:04.035038 7873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:04.042344 7873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:04.042492 7873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:04.042658 7873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.1
I20260811 07:01:04.051712 7873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.40.1
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.1
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:04.053014 7873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:04.054854 7873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:04.068439 7884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:04.068662 7885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:04.069141 7887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:04.069656 7873 server_base.cc:1034] running on GCE node
I20260811 07:01:04.070363 7873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:04.071424 7873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:04.072664 7873 hybrid_clock.cc:648] HybridClock initialized: now 1786431664072617 us; error 52 us; skew 500 ppm
I20260811 07:01:04.072989 7873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:01:04.075238 7873 webserver.cc:460] Webserver started at http://127.4.40.1:34369/ using document root <none> and password file <none>
I20260811 07:01:04.075953 7873 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:04.076139 7873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:04.076503 7873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:04.079366 7873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data/instance:
uuid: "2f5a669f32eb46058cdb8950eb44c4ff"
format_stamp: "Formatted at 2026-08-11 07:01:04 on dist-test-slave-7vjv"
server_key: "11c0ff92fa60e0cd740e962e58c345cb"
server_key_iv: "6e2a830d16ffc91ec354c315a665a7b0"
server_key_version: "encryptionkey@0"
I20260811 07:01:04.080305 7873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance:
uuid: "2f5a669f32eb46058cdb8950eb44c4ff"
format_stamp: "Formatted at 2026-08-11 07:01:04 on dist-test-slave-7vjv"
server_key: "11c0ff92fa60e0cd740e962e58c345cb"
server_key_iv: "6e2a830d16ffc91ec354c315a665a7b0"
server_key_version: "encryptionkey@0"
I20260811 07:01:04.085549 7873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 07:01:04.089010 7897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:04.090272 7873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:01:04.090454 7873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal
uuid: "2f5a669f32eb46058cdb8950eb44c4ff"
format_stamp: "Formatted at 2026-08-11 07:01:04 on dist-test-slave-7vjv"
server_key: "11c0ff92fa60e0cd740e962e58c345cb"
server_key_iv: "6e2a830d16ffc91ec354c315a665a7b0"
server_key_version: "encryptionkey@0"
I20260811 07:01:04.090680 7873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:04.107790 7873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:04.108686 7873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:04.108989 7873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:04.109987 7873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:01:04.111498 7873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:01:04.111639 7873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:04.111764 7873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:01:04.111855 7873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:04.145543 7873 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.1:37779
I20260811 07:01:04.145633 8099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.1:37779 every 8 connection(s)
I20260811 07:01:04.146952 7873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/data/info.pb
I20260811 07:01:04.148371 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 7873
I20260811 07:01:04.148551 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal/instance
I20260811 07:01:04.149114 4256 external_mini_cluster.cc:1428] Setting key 3bead5b8d04acae75e24bc0472e96fe1
I20260811 07:01:04.155614 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.2:0
--local_ip_for_outbound_sockets=127.4.40.2
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45985
--builtin_ntp_servers=127.4.40.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:01:04.160362 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:04.149942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:53193 (local address 127.4.40.62:45985)
0811 07:01:04.153770 (+ 3828us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:04.153775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:04.153790 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:01:04.153838 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:04.153841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:04.153905 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:04.154031 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:04.155084 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.155868 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.157459 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.157873 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.158018 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:04.159432 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:04.159444 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:04.159448 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:04.159466 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:04.159505 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:04.159535 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:04.159652 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:04.159872 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 07:01:04.159975 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3725,"thread_start_us":3636,"threads_started":1}
I20260811 07:01:04.161275 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:04.150391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45985 (local address 127.4.40.1:53193)
0811 07:01:04.151548 (+ 1157us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:04.151603 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:04.153270 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:04.154270 (+ 1000us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:04.154278 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:04.154361 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:04.154918 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:04.154934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.156023 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.156034 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:04.157295 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:04.157307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.158105 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.158113 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:04.158345 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:04.159128 (+ 783us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:01:04.159131 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:04.159168 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:01:04.159237 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:04.159651 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:04.159666 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:04.159675 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 07:01:04.159858 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 07:01:04.160254 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":379,"threads_started":1}
I20260811 07:01:04.162658 8104 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45985
I20260811 07:01:04.162946 8104 heartbeater.cc:464] Registering TS with master...
I20260811 07:01:04.163710 8104 heartbeater.cc:511] Master 127.4.40.62:45985 requested a full tablet report, sending...
I20260811 07:01:04.165768 7758 ts_manager.cc:194] Registered new tserver with Master: 2f5a669f32eb46058cdb8950eb44c4ff (127.4.40.1:37779)
I20260811 07:01:04.167333 7758 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.1:53193
W20260811 07:01:04.526093 8110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:04.526373 8110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:04.526530 8110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:04.533936 8110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:04.534041 8110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:04.534216 8110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.2
I20260811 07:01:04.546173 8110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.40.2
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.2
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:04.547631 8110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:04.549912 8110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:04.565086 8118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:04.565120 8121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:04.565131 8119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:04.566287 8110 server_base.cc:1034] running on GCE node
I20260811 07:01:04.566913 8110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:04.567893 8110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:04.569104 8110 hybrid_clock.cc:648] HybridClock initialized: now 1786431664569051 us; error 42 us; skew 500 ppm
I20260811 07:01:04.569445 8110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:01:04.571750 8110 webserver.cc:460] Webserver started at http://127.4.40.2:37009/ using document root <none> and password file <none>
I20260811 07:01:04.572580 8110 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:04.572715 8110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:04.573110 8110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:04.575974 8110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data/instance:
uuid: "2c154307f94a4a2d90369ec73c990d94"
format_stamp: "Formatted at 2026-08-11 07:01:04 on dist-test-slave-7vjv"
server_key: "8ed9e790c4afebf832201b05c3efbec9"
server_key_iv: "080a6299759bb9b2132300ae1b014bf2"
server_key_version: "encryptionkey@0"
I20260811 07:01:04.576804 8110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance:
uuid: "2c154307f94a4a2d90369ec73c990d94"
format_stamp: "Formatted at 2026-08-11 07:01:04 on dist-test-slave-7vjv"
server_key: "8ed9e790c4afebf832201b05c3efbec9"
server_key_iv: "080a6299759bb9b2132300ae1b014bf2"
server_key_version: "encryptionkey@0"
I20260811 07:01:04.582078 8110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 07:01:04.586086 8131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:04.587379 8110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 07:01:04.587575 8110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal
uuid: "2c154307f94a4a2d90369ec73c990d94"
format_stamp: "Formatted at 2026-08-11 07:01:04 on dist-test-slave-7vjv"
server_key: "8ed9e790c4afebf832201b05c3efbec9"
server_key_iv: "080a6299759bb9b2132300ae1b014bf2"
server_key_version: "encryptionkey@0"
I20260811 07:01:04.587802 8110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:04.603214 8110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:04.604048 8110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:04.604411 8110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:04.605389 8110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:01:04.606871 8110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:01:04.606984 8110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:04.607122 8110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:01:04.607219 8110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:04.641609 8110 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.2:45679
I20260811 07:01:04.641692 8341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.2:45679 every 8 connection(s)
I20260811 07:01:04.642962 8110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/data/info.pb
I20260811 07:01:04.652887 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8110
I20260811 07:01:04.653083 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal/instance
I20260811 07:01:04.653680 4256 external_mini_cluster.cc:1428] Setting key a4f3cdbaee85c1d2180a312fe9c594e3
I20260811 07:01:04.655810 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:04.645644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.2:58955 (local address 127.4.40.62:45985)
0811 07:01:04.645832 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:04.645837 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:04.647161 (+ 1324us) server_negotiation.cc:407] Connection header received
0811 07:01:04.649060 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:04.649066 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:04.649156 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:04.649274 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:04.650346 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.651062 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.652718 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.653217 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.653364 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:04.654981 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:04.654987 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:04.654991 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:04.655012 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:04.655050 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:04.655066 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:04.655162 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:04.655362 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 07:01:04.655450 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 07:01:04.657030 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:04.646128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45985 (local address 127.4.40.2:58955)
0811 07:01:04.647008 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:04.647045 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:04.648778 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:04.649514 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:04.649523 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:04.649606 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:04.650171 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:04.650188 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.651218 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.651226 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:04.652539 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:04.652555 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:04.653458 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:04.653465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:04.653798 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:04.654692 (+ 894us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:01:04.654695 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:04.654733 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:01:04.654808 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:04.655167 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:04.655175 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:04.655181 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:01:04.655323 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 07:01:04.655679 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":296,"threads_started":1}
I20260811 07:01:04.657291 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.40.3:0
--local_ip_for_outbound_sockets=127.4.40.3
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45985
--builtin_ntp_servers=127.4.40.20:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 07:01:04.658485 8345 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45985
I20260811 07:01:04.658807 8345 heartbeater.cc:464] Registering TS with master...
I20260811 07:01:04.659752 8345 heartbeater.cc:511] Master 127.4.40.62:45985 requested a full tablet report, sending...
I20260811 07:01:04.661312 7758 ts_manager.cc:194] Registered new tserver with Master: 2c154307f94a4a2d90369ec73c990d94 (127.4.40.2:45679)
I20260811 07:01:04.662271 7758 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.2:58955
W20260811 07:01:05.006268 8348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:05.006609 8348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:05.006803 8348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:05.013999 8348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:05.014132 8348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:05.014317 8348 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.40.3
I20260811 07:01:05.023070 8348 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.40.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.40.3
--webserver_port=0
--tserver_master_addrs=127.4.40.62:45985
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.40.3
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:05.024446 8348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:05.026413 8348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:05.039768 8362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:05.039863 8356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:05.039878 8355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:05.040807 8348 server_base.cc:1034] running on GCE node
I20260811 07:01:05.041401 8348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:05.042398 8348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:05.043606 8348 hybrid_clock.cc:648] HybridClock initialized: now 1786431665043571 us; error 34 us; skew 500 ppm
I20260811 07:01:05.043968 8348 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 07:01:05.046407 8348 webserver.cc:460] Webserver started at http://127.4.40.3:45977/ using document root <none> and password file <none>
I20260811 07:01:05.047119 8348 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:05.047245 8348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:05.047605 8348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:05.050920 8348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data/instance:
uuid: "2384583b18f149cb8e308eed3ebed033"
format_stamp: "Formatted at 2026-08-11 07:01:05 on dist-test-slave-7vjv"
server_key: "fa99f155547f22061a46fc4ff17c88ee"
server_key_iv: "c89ff8b4615e0c63bf718ea5d6b49948"
server_key_version: "encryptionkey@0"
I20260811 07:01:05.051749 8348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance:
uuid: "2384583b18f149cb8e308eed3ebed033"
format_stamp: "Formatted at 2026-08-11 07:01:05 on dist-test-slave-7vjv"
server_key: "fa99f155547f22061a46fc4ff17c88ee"
server_key_iv: "c89ff8b4615e0c63bf718ea5d6b49948"
server_key_version: "encryptionkey@0"
I20260811 07:01:05.056841 8348 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 07:01:05.060323 8373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:05.061653 8348 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:01:05.061818 8348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal
uuid: "2384583b18f149cb8e308eed3ebed033"
format_stamp: "Formatted at 2026-08-11 07:01:05 on dist-test-slave-7vjv"
server_key: "fa99f155547f22061a46fc4ff17c88ee"
server_key_iv: "c89ff8b4615e0c63bf718ea5d6b49948"
server_key_version: "encryptionkey@0"
I20260811 07:01:05.062049 8348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:05.095896 8348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:05.096872 8348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:05.097177 8348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:05.098176 8348 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 07:01:05.099668 8348 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 07:01:05.099766 8348 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:05.099923 8348 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 07:01:05.100024 8348 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:05.133132 8348 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.3:44647
I20260811 07:01:05.133226 8565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.3:44647 every 8 connection(s)
I20260811 07:01:05.134510 8348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/data/info.pb
I20260811 07:01:05.142392 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8348
I20260811 07:01:05.142570 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal/instance
I20260811 07:01:05.143033 4256 external_mini_cluster.cc:1428] Setting key d0b3db7f7e55082c306cd665db56a2c4
I20260811 07:01:05.147552 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.137190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.3:45867 (local address 127.4.40.62:45985)
0811 07:01:05.137416 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.137423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.138783 (+ 1360us) server_negotiation.cc:407] Connection header received
0811 07:01:05.140574 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.140578 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.140646 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.140734 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:05.141711 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.142481 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.144377 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.145017 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.145158 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.146700 (+ 1542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:05.146707 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:05.146713 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:05.146734 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:05.146773 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:05.146788 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:05.146878 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.147090 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.147188 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 07:01:05.148406 8568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.137737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:45985 (local address 127.4.40.3:45867)
0811 07:01:05.138627 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.138667 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:05.140344 (+ 1677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:05.140928 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:05.140936 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:05.141015 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:05.141570 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.141586 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.142698 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.142703 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.144202 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.144227 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.145279 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.145286 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.145583 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.146403 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 07:01:05.146405 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:05.146442 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 07:01:05.146513 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:05.146884 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:05.146892 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:05.146898 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 07:01:05.147040 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 07:01:05.147302 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":287,"threads_started":1}
I20260811 07:01:05.149713 8566 heartbeater.cc:347] Connected to a master server at 127.4.40.62:45985
I20260811 07:01:05.150012 8566 heartbeater.cc:464] Registering TS with master...
I20260811 07:01:05.150715 8566 heartbeater.cc:511] Master 127.4.40.62:45985 requested a full tablet report, sending...
I20260811 07:01:05.151980 7758 ts_manager.cc:194] Registered new tserver with Master: 2384583b18f149cb8e308eed3ebed033 (127.4.40.3:44647)
I20260811 07:01:05.152809 7758 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.40.3:45867
I20260811 07:01:05.158403 4256 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 07:01:05.169102 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.162962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.4.40.62:45985)
0811 07:01:05.163124 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.163128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.163673 (+ 545us) server_negotiation.cc:407] Connection header received
0811 07:01:05.163816 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.163820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.163885 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.163963 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:05.164613 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.165310 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.166994 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.167384 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.167481 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.168533 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:05.168537 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:05.168540 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:05.168557 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:05.168588 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:05.168600 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:05.168670 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.168798 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.168868 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:01:05.170439 8104 heartbeater.cc:503] Master 127.4.40.62:45985 was elected leader, sending a full tablet report...
I20260811 07:01:05.173172 7758 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54590:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 07:01:05.179832 7758 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 07:01:05.214890 8585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.192768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44647 (local address 127.0.0.1:41314)
0811 07:01:05.205631 (+ 12863us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.205656 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:05.205834 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:05.207439 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:05.207449 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:05.207482 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:05.207903 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.207921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.210137 (+ 2216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.210142 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.212151 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.212166 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.213931 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.213937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.214122 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.214175 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 07:01:05.214305 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 07:01:05.214501 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12704,"mutex_wait_us":87,"thread_start_us":13179,"threads_started":1}
I20260811 07:01:05.214881 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.193074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41314 (local address 127.4.40.3:44647)
0811 07:01:05.206673 (+ 13599us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.206682 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.206714 (+ 32us) server_negotiation.cc:407] Connection header received
0811 07:01:05.206827 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.206836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.207156 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.207431 (+ 275us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:05.208206 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.209983 (+ 1777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.212368 (+ 2385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.213740 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.213903 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.214201 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.214359 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.214520 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13429,"thread_start_us":13265,"threads_started":1}
I20260811 07:01:05.216332 8579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.191440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:45679 (local address 127.0.0.1:59986)
0811 07:01:05.192191 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.192226 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:05.192393 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:05.207978 (+ 15585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:05.207985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:05.208014 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:05.208515 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.208535 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.210359 (+ 1824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.210364 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.213030 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.213046 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.215286 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.215293 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.215553 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.215694 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 07:01:05.215777 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 07:01:05.215956 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":263,"thread_start_us":340,"threads_started":1}
I20260811 07:01:05.216444 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.191690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59986 (local address 127.4.40.2:45679)
0811 07:01:05.207392 (+ 15702us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.207400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.207429 (+ 29us) server_negotiation.cc:407] Connection header received
0811 07:01:05.207537 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.207546 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.207762 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.207986 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:05.208690 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.210210 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.213293 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.214762 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.214897 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.215130 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.215881 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.216062 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15555,"thread_start_us":15429,"threads_started":1}
I20260811 07:01:05.216606 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.192248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42470 (local address 127.4.40.1:37779)
0811 07:01:05.199067 (+ 6819us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.199080 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.205591 (+ 6511us) server_negotiation.cc:407] Connection header received
0811 07:01:05.207032 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.207040 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.207242 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.207431 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:05.208492 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.210082 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.213303 (+ 3221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.214762 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.214897 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.215165 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.215995 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.216217 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6571,"thread_start_us":6390,"threads_started":1}
I20260811 07:01:05.217227 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.191607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42470)
0811 07:01:05.205405 (+ 13798us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.205436 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:05.206749 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:05.207742 (+ 993us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:05.207746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:05.207770 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:05.208261 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.208280 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.210264 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.210267 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.213079 (+ 2812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.213092 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.215298 (+ 2206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.215303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.215550 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.215696 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 07:01:05.215774 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 07:01:05.216978 (+ 1204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13566,"mutex_wait_us":1293,"thread_start_us":14024,"threads_started":1}
I20260811 07:01:05.219693 8453 tablet_service.cc:1467] Processing CreateTablet for tablet 256e00256ff64c8eafdc203983c274d2 (DEFAULT_TABLE table=test-table [id=ae36be91703e4c6da169e18d67438e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:01:05.219753 8225 tablet_service.cc:1467] Processing CreateTablet for tablet 256e00256ff64c8eafdc203983c274d2 (DEFAULT_TABLE table=test-table [id=ae36be91703e4c6da169e18d67438e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:01:05.221098 8225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 256e00256ff64c8eafdc203983c274d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:05.221249 8453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 256e00256ff64c8eafdc203983c274d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:05.220803 7989 tablet_service.cc:1467] Processing CreateTablet for tablet 256e00256ff64c8eafdc203983c274d2 (DEFAULT_TABLE table=test-table [id=ae36be91703e4c6da169e18d67438e73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 07:01:05.222208 7989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 256e00256ff64c8eafdc203983c274d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:05.232158 8589 tablet_bootstrap.cc:492] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Bootstrap starting.
I20260811 07:01:05.236361 8589 tablet_bootstrap.cc:654] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:05.236554 8590 tablet_bootstrap.cc:492] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Bootstrap starting.
I20260811 07:01:05.238231 8589 log.cc:824] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:05.240638 8590 tablet_bootstrap.cc:654] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:05.241955 8590 log.cc:824] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:05.242820 8591 tablet_bootstrap.cc:492] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Bootstrap starting.
I20260811 07:01:05.243093 8589 tablet_bootstrap.cc:492] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: No bootstrap required, opened a new log
I20260811 07:01:05.243391 8589 ts_tablet_manager.cc:1397] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260811 07:01:05.246008 8590 tablet_bootstrap.cc:492] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: No bootstrap required, opened a new log
I20260811 07:01:05.246272 8590 ts_tablet_manager.cc:1397] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 07:01:05.247612 8591 tablet_bootstrap.cc:654] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:05.248864 8591 log.cc:824] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:05.251904 8591 tablet_bootstrap.cc:492] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: No bootstrap required, opened a new log
I20260811 07:01:05.252278 8591 ts_tablet_manager.cc:1397] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 07:01:05.254406 8590 raft_consensus.cc:348] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.254285 8589 raft_consensus.cc:348] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.254854 8589 raft_consensus.cc:374] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:05.254969 8590 raft_consensus.cc:374] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:05.255064 8589 raft_consensus.cc:729] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c154307f94a4a2d90369ec73c990d94, State: Initialized, Role: FOLLOWER
I20260811 07:01:05.255160 8590 raft_consensus.cc:729] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f5a669f32eb46058cdb8950eb44c4ff, State: Initialized, Role: FOLLOWER
I20260811 07:01:05.255820 8590 consensus_queue.cc:260] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.255800 8589 consensus_queue.cc:260] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.257385 8590 ts_tablet_manager.cc:1428] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 07:01:05.261118 8345 heartbeater.cc:503] Master 127.4.40.62:45985 was elected leader, sending a full tablet report...
I20260811 07:01:05.261965 8589 ts_tablet_manager.cc:1428] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Time spent starting tablet: real 0.018s user 0.014s sys 0.004s
I20260811 07:01:05.267413 8591 raft_consensus.cc:348] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.267788 8591 raft_consensus.cc:374] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:05.267926 8591 raft_consensus.cc:729] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2384583b18f149cb8e308eed3ebed033, State: Initialized, Role: FOLLOWER
I20260811 07:01:05.268568 8591 consensus_queue.cc:260] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.269621 8566 heartbeater.cc:503] Master 127.4.40.62:45985 was elected leader, sending a full tablet report...
I20260811 07:01:05.270190 8591 ts_tablet_manager.cc:1428] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Time spent starting tablet: real 0.018s user 0.014s sys 0.000s
I20260811 07:01:05.272065 8598 raft_consensus.cc:482] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 07:01:05.272411 8598 raft_consensus.cc:504] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.274410 8598 leader_election.cc:288] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2c154307f94a4a2d90369ec73c990d94 (127.4.40.2:45679), 2384583b18f149cb8e308eed3ebed033 (127.4.40.3:44647)
I20260811 07:01:05.281203 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.274927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:56349 (local address 127.4.40.3:44647)
0811 07:01:05.275158 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.275164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.275761 (+ 597us) server_negotiation.cc:407] Connection header received
0811 07:01:05.275903 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.275915 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.275968 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.276072 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:05.276729 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.277399 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.279327 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.280255 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.280394 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.280497 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.280853 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.280968 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 07:01:05.281268 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.274711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.40.1:59689 (local address 127.4.40.2:45679)
0811 07:01:05.275025 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.275034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.275499 (+ 465us) server_negotiation.cc:407] Connection header received
0811 07:01:05.275719 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.275724 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.275813 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.275914 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 07:01:05.276603 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.277320 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.279311 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.280227 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.280361 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.280453 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.280891 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.281015 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260811 07:01:05.281389 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.274649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.2:45679 (local address 127.4.40.1:59689)
0811 07:01:05.275359 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.275386 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:05.275567 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:05.275943 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:05.275948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:05.275983 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:05.276443 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.276453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.277451 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.277457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.279108 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.279121 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.280411 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.280417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.280590 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.280736 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 07:01:05.280854 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 07:01:05.280979 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":101,"spinlock_wait_cycles":27008,"thread_start_us":390,"threads_started":1}
I20260811 07:01:05.281555 8609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.274742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.3:44647 (local address 127.4.40.1:56349)
0811 07:01:05.275681 (+ 939us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.275698 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:05.275805 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:05.276116 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:05.276121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:05.276160 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 07:01:05.276517 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.276526 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.277546 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.277552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.279110 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:05.279123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.280448 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.280454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:05.280687 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.280710 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 07:01:05.280827 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 07:01:05.280951 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":81,"thread_start_us":596,"threads_started":1}
I20260811 07:01:05.282444 8493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "256e00256ff64c8eafdc203983c274d2" candidate_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2384583b18f149cb8e308eed3ebed033" is_pre_election: true
I20260811 07:01:05.282451 8260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "256e00256ff64c8eafdc203983c274d2" candidate_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c154307f94a4a2d90369ec73c990d94" is_pre_election: true
I20260811 07:01:05.282923 8493 raft_consensus.cc:2454] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f5a669f32eb46058cdb8950eb44c4ff in term 0.
I20260811 07:01:05.282980 8260 raft_consensus.cc:2454] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f5a669f32eb46058cdb8950eb44c4ff in term 0.
I20260811 07:01:05.283696 7904 leader_election.cc:302] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2384583b18f149cb8e308eed3ebed033, 2c154307f94a4a2d90369ec73c990d94, 2f5a669f32eb46058cdb8950eb44c4ff; no voters:
I20260811 07:01:05.284216 8598 raft_consensus.cc:2781] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 07:01:05.284387 8598 raft_consensus.cc:482] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 07:01:05.284525 8598 raft_consensus.cc:3037] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:05.286804 8598 raft_consensus.cc:504] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.287415 8598 leader_election.cc:288] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [CANDIDATE]: Term 1 election: Requested vote from peers 2c154307f94a4a2d90369ec73c990d94 (127.4.40.2:45679), 2384583b18f149cb8e308eed3ebed033 (127.4.40.3:44647)
I20260811 07:01:05.287884 8493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "256e00256ff64c8eafdc203983c274d2" candidate_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2384583b18f149cb8e308eed3ebed033"
I20260811 07:01:05.287904 8260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "256e00256ff64c8eafdc203983c274d2" candidate_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c154307f94a4a2d90369ec73c990d94"
I20260811 07:01:05.288044 8493 raft_consensus.cc:3037] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:05.288101 8260 raft_consensus.cc:3037] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:05.290316 8493 raft_consensus.cc:2454] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f5a669f32eb46058cdb8950eb44c4ff in term 1.
I20260811 07:01:05.290370 8260 raft_consensus.cc:2454] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f5a669f32eb46058cdb8950eb44c4ff in term 1.
I20260811 07:01:05.290710 7901 leader_election.cc:302] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2384583b18f149cb8e308eed3ebed033, 2f5a669f32eb46058cdb8950eb44c4ff; no voters:
I20260811 07:01:05.291028 8598 raft_consensus.cc:2781] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:05.291498 8598 raft_consensus.cc:686] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 1 LEADER]: Becoming Leader. State: Replica: 2f5a669f32eb46058cdb8950eb44c4ff, State: Running, Role: LEADER
I20260811 07:01:05.292063 8598 consensus_queue.cc:237] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } }
I20260811 07:01:05.296752 7758 catalog_manager.cc:5495] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f5a669f32eb46058cdb8950eb44c4ff (127.4.40.1). New cstate: current_term: 1 leader_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f5a669f32eb46058cdb8950eb44c4ff" member_type: VOTER last_known_addr { host: "127.4.40.1" port: 37779 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 } health_report { overall_health: UNKNOWN } } }
I20260811 07:01:05.311498 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.306545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42480 (local address 127.4.40.1:37779)
0811 07:01:05.306745 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.306750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.306767 (+ 17us) server_negotiation.cc:407] Connection header received
0811 07:01:05.306918 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.306922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.307007 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.307135 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:05.307436 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.308186 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.309570 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.309965 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.310054 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.310473 (+ 419us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:05.310951 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:05.311060 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.311153 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.311214 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 07:01:05.319573 8493 raft_consensus.cc:1261] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 1 FOLLOWER]: Refusing update from remote peer 2f5a669f32eb46058cdb8950eb44c4ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:05.319572 8260 raft_consensus.cc:1261] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 1 FOLLOWER]: Refusing update from remote peer 2f5a669f32eb46058cdb8950eb44c4ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:05.320614 8598 consensus_queue.cc:1035] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2384583b18f149cb8e308eed3ebed033" member_type: VOTER last_known_addr { host: "127.4.40.3" port: 44647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:05.320910 8610 consensus_queue.cc:1035] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c154307f94a4a2d90369ec73c990d94" member_type: VOTER last_known_addr { host: "127.4.40.2" port: 45679 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:05.332875 8615 mvcc.cc:205] Tried to move back new op lower bound from 7317224101139058688 to 7317224101040726016. Current Snapshot: MvccSnapshot[applied={T|T < 7317224101139058688}]
I20260811 07:01:05.334522 8622 mvcc.cc:205] Tried to move back new op lower bound from 7317224101139058688 to 7317224101040726016. Current Snapshot: MvccSnapshot[applied={T|T < 7317224101139058688}]
I20260811 07:01:05.347209 7758 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:01:05.347469 7758 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54590:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 07:01:05.351024 7758 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3582db3e119644e695fe6f27ce3a3ed8: Sending DeleteTablet for 3 replicas of tablet 256e00256ff64c8eafdc203983c274d2
I20260811 07:01:05.352334 8453 tablet_service.cc:1514] Processing DeleteTablet for tablet 256e00256ff64c8eafdc203983c274d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:01:05 UTC) from {username='slave'} at 127.0.0.1:41314
I20260811 07:01:05.352391 8225 tablet_service.cc:1514] Processing DeleteTablet for tablet 256e00256ff64c8eafdc203983c274d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:01:05 UTC) from {username='slave'} at 127.0.0.1:59986
I20260811 07:01:05.353399 7989 tablet_service.cc:1514] Processing DeleteTablet for tablet 256e00256ff64c8eafdc203983c274d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 07:01:05 UTC) from {username='slave'} at 127.0.0.1:42470
I20260811 07:01:05.353765 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7873
I20260811 07:01:05.359678 8650 tablet_replica.cc:330] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: stopping tablet replica
I20260811 07:01:05.360096 8650 raft_consensus.cc:2229] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:01:05.360636 8650 raft_consensus.cc:2258] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:01:05.363286 8650 ts_tablet_manager.cc:1905] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:01:05.372905 8650 ts_tablet_manager.cc:1918] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:01:05.373191 8650 log.cc:1196] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-2/wal/wals/256e00256ff64c8eafdc203983c274d2
I20260811 07:01:05.373965 8650 ts_tablet_manager.cc:1939] T 256e00256ff64c8eafdc203983c274d2 P 2384583b18f149cb8e308eed3ebed033: Deleting consensus metadata
I20260811 07:01:05.376617 7732 catalog_manager.cc:4841] TS 2384583b18f149cb8e308eed3ebed033 (127.4.40.3:44647): tablet 256e00256ff64c8eafdc203983c274d2 (table test-table [id=ae36be91703e4c6da169e18d67438e73]) successfully deleted
I20260811 07:01:05.378875 8658 tablet_replica.cc:330] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: stopping tablet replica
I20260811 07:01:05.379365 8658 raft_consensus.cc:2229] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 1 LEADER]: Raft consensus shutting down.
I20260811 07:01:05.380270 8658 raft_consensus.cc:2258] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:01:05.382611 8658 ts_tablet_manager.cc:1905] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:01:05.389400 8584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.356273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42490 (local address 127.4.40.1:37779)
0811 07:01:05.356477 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.356484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.378027 (+ 21543us) server_negotiation.cc:407] Connection header received
0811 07:01:05.378251 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.378260 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.378398 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.378530 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:05.379154 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.380889 (+ 1735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.383597 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.384636 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.384779 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.386842 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:05.386857 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:05.386879 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:05.387992 (+ 1113us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:05.388113 (+ 121us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:05.388147 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:05.388277 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.388604 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.388771 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:01:05.390854 8094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:05.391856 8656 tablet_replica.cc:330] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: stopping tablet replica
I20260811 07:01:05.392580 8656 raft_consensus.cc:2229] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 07:01:05.393252 8656 raft_consensus.cc:2258] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 07:01:05.396147 8656 ts_tablet_manager.cc:1905] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 07:01:05.400986 8656 ts_tablet_manager.cc:1918] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:01:05.401142 8656 log.cc:1196] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-1/wal/wals/256e00256ff64c8eafdc203983c274d2
I20260811 07:01:05.401645 8656 ts_tablet_manager.cc:1939] T 256e00256ff64c8eafdc203983c274d2 P 2c154307f94a4a2d90369ec73c990d94: Deleting consensus metadata
I20260811 07:01:05.403189 7733 catalog_manager.cc:4841] TS 2c154307f94a4a2d90369ec73c990d94 (127.4.40.2:45679): tablet 256e00256ff64c8eafdc203983c274d2 (table test-table [id=ae36be91703e4c6da169e18d67438e73]) successfully deleted
I20260811 07:01:05.489920 8658 ts_tablet_manager.cc:1918] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 07:01:05.490291 8658 log.cc:1196] T 256e00256ff64c8eafdc203983c274d2 P 2f5a669f32eb46058cdb8950eb44c4ff: Deleting WAL directory at /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786431333861527-4256-0/minicluster-data/ts-0/wal/wals/256e00256ff64c8eafdc203983c274d2
I20260811 07:01:05.490641 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7873
W20260811 07:01:05.517195 7730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.40.1:37779 (error 108)
I20260811 07:01:05.517446 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8110
I20260811 07:01:05.519270 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.518846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42492)
0811 07:01:05.519014 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.519124 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 07:01:05.519606 7730 catalog_manager.cc:4584] TS 2f5a669f32eb46058cdb8950eb44c4ff (127.4.40.1:37779): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 256e00256ff64c8eafdc203983c274d2: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
I20260811 07:01:05.524497 8581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.518406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59996 (local address 127.4.40.2:45679)
0811 07:01:05.518590 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.518596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.518639 (+ 43us) server_negotiation.cc:407] Connection header received
0811 07:01:05.518863 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.518869 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.518981 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.519104 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:05.519560 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.520524 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.521756 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.522147 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.522244 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.523177 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:05.523186 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:05.523200 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:05.523759 (+ 559us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:05.523805 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:05.523824 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:05.523918 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.524043 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.524228 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 07:01:05.525183 8335 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:05.573549 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.572679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42498)
0811 07:01:05.573163 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.573293 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314}
I20260811 07:01:05.574836 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.574442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42508)
0811 07:01:05.574637 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.574705 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 07:01:05.615302 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8110
I20260811 07:01:05.635672 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.635045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42524)
0811 07:01:05.635314 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.635433 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 07:01:05.636873 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.636391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42538)
0811 07:01:05.636578 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.636679 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 07:01:05.646867 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8348
I20260811 07:01:05.653244 8587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.647613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41330 (local address 127.4.40.3:44647)
0811 07:01:05.647773 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:05.647777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:05.647795 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:01:05.647895 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:05.647900 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:05.648004 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:05.648139 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:05.648527 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.649264 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.650496 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:05.650892 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:05.650980 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:05.651943 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:05.651952 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:05.651966 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:05.652627 (+ 661us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:05.652674 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:05.652695 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:05.652806 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:05.652924 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 07:01:05.653010 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 07:01:05.653893 8561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:05.717561 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.717029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42554)
0811 07:01:05.717273 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.717381 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 07:01:05.718699 8580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:05.718234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.1:37779 (local address 127.0.0.1:42570)
0811 07:01:05.718462 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:05.718548 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.1:37779: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260811 07:01:05.743647 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8348
I20260811 07:01:05.770188 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 7704
I20260811 07:01:05.771355 7824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:05.881973 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 7704
2026-08-11T07:01:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2970 ms)
[----------] 6 tests from EncryptionPolicyTest (23583 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:06 dist-test-slave-7vjv krb5kdc[8681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:06 dist-test-slave-7vjv krb5kdc[8681](info): set up 2 sockets
Aug 11 07:01:06 dist-test-slave-7vjv krb5kdc[8681](info): commencing operation
krb5kdc: starting...
W20260811 07:01:07.282522 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:07 dist-test-slave-7vjv krb5kdc[8681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:07.619374 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39917
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:07.965148 8707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:07.965426 8707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:07.965528 8707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:07.972607 8707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:07.972708 8707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:07.972752 8707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:07.972820 8707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:07.972864 8707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:07.982441 8707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39917
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:07.983626 8707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:07.984931 8707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:07.996734 8713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:07.996701 8715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:07.996943 8718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:07.997785 8707 server_base.cc:1034] running on GCE node
I20260811 07:01:07.998536 8707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:07.999938 8707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:08.001180 8707 hybrid_clock.cc:648] HybridClock initialized: now 1786431668001136 us; error 54 us; skew 500 ppm
I20260811 07:01:08.001531 8707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:08 dist-test-slave-7vjv krb5kdc[8681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431668, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:08.009105 8707 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:08.010681 8707 webserver.cc:460] Webserver started at http://127.0.0.1:39865/ using document root <none> and password file <none>
I20260811 07:01:08.011402 8707 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:08.011554 8707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:08.011862 8707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:08.015012 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "2abf8e34dc35498fad3dcd5fa5addafd"
format_stamp: "Formatted at 2026-08-11 07:01:08 on dist-test-slave-7vjv"
server_key: "2f313429ab87ca7b24cfd54814325b60"
server_key_iv: "db699bf49976f54fe5d1230c5e50e62f"
server_key_version: "encryptionkey@0"
I20260811 07:01:08.015791 8707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "2abf8e34dc35498fad3dcd5fa5addafd"
format_stamp: "Formatted at 2026-08-11 07:01:08 on dist-test-slave-7vjv"
server_key: "2f313429ab87ca7b24cfd54814325b60"
server_key_iv: "db699bf49976f54fe5d1230c5e50e62f"
server_key_version: "encryptionkey@0"
I20260811 07:01:08.020916 8707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.003s
I20260811 07:01:08.024289 8731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:08.025548 8707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:01:08.025758 8707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "2abf8e34dc35498fad3dcd5fa5addafd"
format_stamp: "Formatted at 2026-08-11 07:01:08 on dist-test-slave-7vjv"
server_key: "2f313429ab87ca7b24cfd54814325b60"
server_key_iv: "db699bf49976f54fe5d1230c5e50e62f"
server_key_version: "encryptionkey@0"
I20260811 07:01:08.025974 8707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:08.048012 8707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:08.051994 8707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:08.052336 8707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:08.070549 8707 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39917
I20260811 07:01:08.070592 8816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39917 every 8 connection(s)
I20260811 07:01:08.072023 8707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:08.075740 8817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:08.082160 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8707
I20260811 07:01:08.082420 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:08.083318 4256 external_mini_cluster.cc:1428] Setting key 051b1e0381ade0510ee5ff623e18714a
I20260811 07:01:08.087285 8817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd: Bootstrap starting.
I20260811 07:01:08.091074 8817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:08.092141 8817 log.cc:824] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:08.095081 8817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd: No bootstrap required, opened a new log
Aug 11 07:01:08 dist-test-slave-7vjv krb5kdc[8681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431667, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:01:08.103295 8817 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39917 } }
I20260811 07:01:08.103667 8817 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:08.103762 8817 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2abf8e34dc35498fad3dcd5fa5addafd, State: Initialized, Role: FOLLOWER
I20260811 07:01:08.104410 8817 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39917 } }
I20260811 07:01:08.104703 8817 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:08.104853 8817 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:08.105065 8817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:08.107151 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:08.085402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32919 (local address 127.0.0.1:39917)
0811 07:01:08.086203 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:08.086213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:08.086254 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:01:08.087931 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:08.087951 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:08.088374 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:08.088696 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:08.090699 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:08.091994 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:08.093591 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:08.094215 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:08.094341 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:08.099154 (+ 4813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:08.099161 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:08.099174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:08.099206 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:08.102449 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:08.103205 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:08.103211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:08.103217 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:08.103380 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:08.103927 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:08.103931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:08.103933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:08.104527 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:08.104893 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:08.105412 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 07:01:08.105767 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":385,"threads_started":1}
I20260811 07:01:08.108186 8817 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39917 } }
I20260811 07:01:08.108680 8817 leader_election.cc:302] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2abf8e34dc35498fad3dcd5fa5addafd; no voters:
I20260811 07:01:08.109282 8817 leader_election.cc:288] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:08.109444 8829 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:08.110924 8829 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [term 1 LEADER]: Becoming Leader. State: Replica: 2abf8e34dc35498fad3dcd5fa5addafd, State: Running, Role: LEADER
I20260811 07:01:08.111644 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:08.111675 8817 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:08.111536 8829 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39917 } }
I20260811 07:01:08.114120 8832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2abf8e34dc35498fad3dcd5fa5addafd. Latest consensus state: current_term: 1 leader_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39917 } } }
I20260811 07:01:08.114483 8832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:08.115027 8830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2abf8e34dc35498fad3dcd5fa5addafd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39917 } } }
I20260811 07:01:08.115360 8830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:08.118952 8839 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:08.124553 8839 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:08.139796 8839 catalog_manager.cc:1347] Generated new cluster ID: 9647f75fed2e420bb9d6e5b8945ce1ca
I20260811 07:01:08.139901 8839 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:08.152045 8826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:08.122855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35085 (local address 127.0.0.1:39917)
0811 07:01:08.126736 (+ 3881us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:08.126745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:08.133662 (+ 6917us) server_negotiation.cc:407] Connection header received
0811 07:01:08.134200 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:08.134208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:08.134307 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:08.134395 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:08.137756 (+ 3361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:08.138898 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:08.140839 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:08.141517 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:08.141606 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:08.144855 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:08.144861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:08.144865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:08.144898 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:08.148289 (+ 3391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:08.149156 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:08.149160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:08.149164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:08.149335 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:08.149825 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:08.149829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:08.149832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:08.150216 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:08.150438 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:08.151536 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0811 07:01:08.151692 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3718}
I20260811 07:01:08.162879 8839 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:08.164587 8839 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:08.174486 8839 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2abf8e34dc35498fad3dcd5fa5addafd: Generated new TSK 0
I20260811 07:01:08.175269 8839 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:08.185046 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8707
I20260811 07:01:08.186650 8809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:08.278393 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8707
W20260811 07:01:08.302029 8673 connection.cc:535] client connection to 127.0.0.1:39917 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39917): Connection reset by peer (error 104)
2026-08-11T07:01:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:08 dist-test-slave-7vjv krb5kdc[8893](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:08 dist-test-slave-7vjv krb5kdc[8893](info): set up 2 sockets
Aug 11 07:01:08 dist-test-slave-7vjv krb5kdc[8893](info): commencing operation
krb5kdc: starting...
W20260811 07:01:09.660410 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.339s user 0.003s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:09 dist-test-slave-7vjv krb5kdc[8893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431669, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:09.993050 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:10.339452 8920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:10.339813 8920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:10.340008 8920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:10.349471 8920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:10.349624 8920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:10.349730 8920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:10.349840 8920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:10.349929 8920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:10.359436 8920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:10.360853 8920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:10.362147 8920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:10.374188 8937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:10.374387 8935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:10.374315 8940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:10.375402 8920 server_base.cc:1034] running on GCE node
I20260811 07:01:10.376251 8920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:10.377630 8920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:10.378863 8920 hybrid_clock.cc:648] HybridClock initialized: now 1786431670378825 us; error 44 us; skew 500 ppm
I20260811 07:01:10.379213 8920 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:10 dist-test-slave-7vjv krb5kdc[8893](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431670, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:10.382522 8920 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:10.384045 8920 webserver.cc:460] Webserver started at http://127.0.0.1:39699/ using document root <none> and password file <none>
I20260811 07:01:10.384783 8920 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:10.384938 8920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:10.385315 8920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:10.388306 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "9c0abd8ee8f04616856a99c7139470fa"
format_stamp: "Formatted at 2026-08-11 07:01:10 on dist-test-slave-7vjv"
server_key: "398294220616a0e2208243d2a64e7fde"
server_key_iv: "5d9381ae5277f7e1487b377ccf58b913"
server_key_version: "encryptionkey@0"
I20260811 07:01:10.389092 8920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "9c0abd8ee8f04616856a99c7139470fa"
format_stamp: "Formatted at 2026-08-11 07:01:10 on dist-test-slave-7vjv"
server_key: "398294220616a0e2208243d2a64e7fde"
server_key_iv: "5d9381ae5277f7e1487b377ccf58b913"
server_key_version: "encryptionkey@0"
I20260811 07:01:10.394130 8920 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 07:01:10.397521 8954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:10.398692 8920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 07:01:10.398895 8920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "9c0abd8ee8f04616856a99c7139470fa"
format_stamp: "Formatted at 2026-08-11 07:01:10 on dist-test-slave-7vjv"
server_key: "398294220616a0e2208243d2a64e7fde"
server_key_iv: "5d9381ae5277f7e1487b377ccf58b913"
server_key_version: "encryptionkey@0"
I20260811 07:01:10.399099 8920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:10.412923 8920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:10.417392 8920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:10.417693 8920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:10.436811 8920 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45699
I20260811 07:01:10.436838 9056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45699 every 8 connection(s)
I20260811 07:01:10.438158 8920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:10.441731 9057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:10.446936 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 8920
I20260811 07:01:10.447147 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:10.447672 4256 external_mini_cluster.cc:1428] Setting key 13a8be082c3c8ac80aa869f88c6455f4
I20260811 07:01:10.452951 9057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa: Bootstrap starting.
I20260811 07:01:10.456014 9057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:10.457090 9057 log.cc:824] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:10.459838 9057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa: No bootstrap required, opened a new log
I20260811 07:01:10.460714 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:10.449592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37357 (local address 127.0.0.1:45699)
0811 07:01:10.450389 (+ 797us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:10.450401 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:10.450448 (+ 47us) server_negotiation.cc:407] Connection header received
0811 07:01:10.452204 (+ 1756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:10.452225 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:10.452576 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:10.452945 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:10.453463 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:10.454793 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:10.456339 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:10.456996 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:10.457213 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:10.458456 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:10.458464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:10.458476 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:10.458960 (+ 484us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:10.459033 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:10.459056 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:10.459217 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:10.459519 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 07:01:10.459717 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":357,"threads_started":1}
I20260811 07:01:10.464479 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:10.467774 9057 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0abd8ee8f04616856a99c7139470fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260811 07:01:10.468145 9057 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:10.468257 9057 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c0abd8ee8f04616856a99c7139470fa, State: Initialized, Role: FOLLOWER
I20260811 07:01:10.468952 9057 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0abd8ee8f04616856a99c7139470fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260811 07:01:10.469298 9057 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:10.469455 9057 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:10.469673 9057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:10.471940 9057 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0abd8ee8f04616856a99c7139470fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260811 07:01:10.472494 9057 leader_election.cc:302] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c0abd8ee8f04616856a99c7139470fa; no voters:
I20260811 07:01:10.473184 9057 leader_election.cc:288] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:10.473315 9075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:10.474073 9075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [term 1 LEADER]: Becoming Leader. State: Replica: 9c0abd8ee8f04616856a99c7139470fa, State: Running, Role: LEADER
I20260811 07:01:10.474862 9075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0abd8ee8f04616856a99c7139470fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } }
I20260811 07:01:10.475726 9060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:10.468889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36285 (local address 127.0.0.1:45699)
0811 07:01:10.469086 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:10.469095 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:10.469558 (+ 463us) server_negotiation.cc:407] Connection header received
0811 07:01:10.469764 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:10.469772 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:10.469841 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:10.469958 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:10.470469 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:10.471273 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:10.472704 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:10.473208 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:10.473381 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:10.474902 (+ 1521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:10.474907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:10.474910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:10.474928 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:10.474964 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:10.474979 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:10.475115 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:10.475388 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 07:01:10.475462 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 07:01:10.478165 9057 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:10.478240 9084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c0abd8ee8f04616856a99c7139470fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0abd8ee8f04616856a99c7139470fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } } }
I20260811 07:01:10.478588 9084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:10.478941 9083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c0abd8ee8f04616856a99c7139470fa. Latest consensus state: current_term: 1 leader_uuid: "9c0abd8ee8f04616856a99c7139470fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c0abd8ee8f04616856a99c7139470fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45699 } } }
I20260811 07:01:10.479287 9083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:10.485378 9088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:10.491659 9088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:10.501031 9088 catalog_manager.cc:1347] Generated new cluster ID: 46b1d4d6b4e84671a510877c4cc0017c
I20260811 07:01:10.501153 9088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:10.523015 9088 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:10.524415 9088 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:10.537997 9088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c0abd8ee8f04616856a99c7139470fa: Generated new TSK 0
I20260811 07:01:10.538832 9088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:10.545611 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8920
W20260811 07:01:10.546743 9048 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37357
W20260811 07:01:10.547619 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 8920: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:10.547828 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 8920
W20260811 07:01:10.572963 8885 connection.cc:535] client connection to 127.0.0.1:45699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45699): Connection reset by peer (error 104)
2026-08-11T07:01:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:10 dist-test-slave-7vjv krb5kdc[9122](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:10 dist-test-slave-7vjv krb5kdc[9122](info): set up 2 sockets
Aug 11 07:01:10 dist-test-slave-7vjv krb5kdc[9122](info): commencing operation
krb5kdc: starting...
W20260811 07:01:11.940639 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.348s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:12 dist-test-slave-7vjv krb5kdc[9122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431672, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:12.271482 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37705
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:12.628177 9149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:12.628499 9149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:12.628664 9149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:12.635768 9149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:12.635928 9149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:12.636054 9149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:12.636190 9149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:12.636276 9149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:12.645846 9149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37705
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:12.647112 9149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:12.648448 9149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:12.660118 9158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:12.660207 9161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:12.660144 9156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:12.661177 9149 server_base.cc:1034] running on GCE node
I20260811 07:01:12.661908 9149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:12.663149 9149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:12.664368 9149 hybrid_clock.cc:648] HybridClock initialized: now 1786431672664327 us; error 27 us; skew 500 ppm
I20260811 07:01:12.664664 9149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:12 dist-test-slave-7vjv krb5kdc[9122](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431672, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:12.667729 9149 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:12.669116 9149 webserver.cc:460] Webserver started at http://127.0.0.1:39539/ using document root <none> and password file <none>
I20260811 07:01:12.669751 9149 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:12.669860 9149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:12.670173 9149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:12.673137 9149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "2505bf8f4d2548258bba3e11076eba9d"
format_stamp: "Formatted at 2026-08-11 07:01:12 on dist-test-slave-7vjv"
server_key: "9b6fd70594779dae414645a9e026825a"
server_key_iv: "4add4b94a63a1fff8a35ed1ef00aa746"
server_key_version: "encryptionkey@0"
I20260811 07:01:12.673834 9149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "2505bf8f4d2548258bba3e11076eba9d"
format_stamp: "Formatted at 2026-08-11 07:01:12 on dist-test-slave-7vjv"
server_key: "9b6fd70594779dae414645a9e026825a"
server_key_iv: "4add4b94a63a1fff8a35ed1ef00aa746"
server_key_version: "encryptionkey@0"
I20260811 07:01:12.679045 9149 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 07:01:12.682516 9175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:12.683737 9149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 07:01:12.683930 9149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "2505bf8f4d2548258bba3e11076eba9d"
format_stamp: "Formatted at 2026-08-11 07:01:12 on dist-test-slave-7vjv"
server_key: "9b6fd70594779dae414645a9e026825a"
server_key_iv: "4add4b94a63a1fff8a35ed1ef00aa746"
server_key_version: "encryptionkey@0"
I20260811 07:01:12.684185 9149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:12.689906 9149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:12.694051 9149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:12.694339 9149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:12.712877 9149 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37705
I20260811 07:01:12.712949 9259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37705 every 8 connection(s)
I20260811 07:01:12.714372 9149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:12.716898 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9149
I20260811 07:01:12.717083 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:12.717556 4256 external_mini_cluster.cc:1428] Setting key b145fd2fbe5db7846b6c6f83ca0ca870
I20260811 07:01:12.718950 9260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:12.725088 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:12.719305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39363 (local address 127.0.0.1:37705)
0811 07:01:12.720073 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:12.720084 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:12.720182 (+ 98us) server_negotiation.cc:407] Connection header received
0811 07:01:12.721713 (+ 1531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:12.721733 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:12.722039 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:12.722408 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:12.723449 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:12.723455 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:12.723474 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:12.723487 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:12.723560 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:12.723579 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:12.723837 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:12.723977 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 07:01:12.724202 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":357,"threads_started":1}
I20260811 07:01:12.728616 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:12.730885 9260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d: Bootstrap starting.
I20260811 07:01:12.734483 9260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:12.735353 9266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:12.733049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41509 (local address 127.0.0.1:37705)
0811 07:01:12.733209 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:12.733215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:12.733519 (+ 304us) server_negotiation.cc:407] Connection header received
0811 07:01:12.733642 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:12.733646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:12.733704 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:12.733817 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:12.734711 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:12.734717 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:12.734721 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:12.734740 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:12.734785 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:12.734805 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:12.734944 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:12.735030 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 07:01:12.735087 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 07:01:12.735810 9260 log.cc:824] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:12.738687 9260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d: No bootstrap required, opened a new log
I20260811 07:01:12.745947 9260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2505bf8f4d2548258bba3e11076eba9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37705 } }
I20260811 07:01:12.746400 9260 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:12.746537 9260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2505bf8f4d2548258bba3e11076eba9d, State: Initialized, Role: FOLLOWER
I20260811 07:01:12.747180 9260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2505bf8f4d2548258bba3e11076eba9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37705 } }
I20260811 07:01:12.747412 9260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:12.747582 9260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:12.747749 9260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:12.749969 9260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2505bf8f4d2548258bba3e11076eba9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37705 } }
I20260811 07:01:12.750465 9260 leader_election.cc:302] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2505bf8f4d2548258bba3e11076eba9d; no voters:
I20260811 07:01:12.751039 9260 leader_election.cc:288] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:12.751147 9274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:12.752486 9274 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [term 1 LEADER]: Becoming Leader. State: Replica: 2505bf8f4d2548258bba3e11076eba9d, State: Running, Role: LEADER
I20260811 07:01:12.753026 9274 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2505bf8f4d2548258bba3e11076eba9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37705 } }
I20260811 07:01:12.753237 9260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:12.755671 9280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2505bf8f4d2548258bba3e11076eba9d. Latest consensus state: current_term: 1 leader_uuid: "2505bf8f4d2548258bba3e11076eba9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2505bf8f4d2548258bba3e11076eba9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37705 } } }
I20260811 07:01:12.756047 9280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:12.756635 9275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2505bf8f4d2548258bba3e11076eba9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2505bf8f4d2548258bba3e11076eba9d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37705 } } }
I20260811 07:01:12.756974 9275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:12.760823 9288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:12.766289 9288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:12.774230 9288 catalog_manager.cc:1347] Generated new cluster ID: d1e03392789c45a696a2d11a67aa43f7
I20260811 07:01:12.774335 9288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:12.787659 9288 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:12.788196 9288 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:12.802208 9288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2505bf8f4d2548258bba3e11076eba9d: Generated new TSK 0
I20260811 07:01:12.802961 9288 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:12.816501 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9149
W20260811 07:01:12.817620 9253 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39363
W20260811 07:01:12.818338 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9149: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:12.818501 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9149
2026-08-11T07:01:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2280 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:12 dist-test-slave-7vjv krb5kdc[9319](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:12 dist-test-slave-7vjv krb5kdc[9319](info): set up 2 sockets
Aug 11 07:01:12 dist-test-slave-7vjv krb5kdc[9319](info): commencing operation
krb5kdc: starting...
W20260811 07:01:14.223377 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.350s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:14 dist-test-slave-7vjv krb5kdc[9319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:14.585595 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:14.929849 9345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:14.930137 9345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:14.930259 9345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:14.937397 9345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:14.937505 9345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:14.937640 9345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:14.937750 9345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:14.937830 9345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:14.947431 9345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:14.948822 9345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:14.950186 9345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:14.962091 9357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:14.962098 9358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:14.962649 9364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:14.963130 9345 server_base.cc:1034] running on GCE node
I20260811 07:01:14.963900 9345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:14.965322 9345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:14.966574 9345 hybrid_clock.cc:648] HybridClock initialized: now 1786431674966521 us; error 45 us; skew 500 ppm
I20260811 07:01:14.966921 9345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:14 dist-test-slave-7vjv krb5kdc[9319](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431674, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:14.970142 9345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:14.971568 9345 webserver.cc:460] Webserver started at http://127.0.0.1:43091/ using document root <none> and password file <none>
I20260811 07:01:14.972302 9345 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:14.972442 9345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:14.972743 9345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:14.975728 9345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "7acb7f418d5e42bb9ce1a43335faaade"
format_stamp: "Formatted at 2026-08-11 07:01:14 on dist-test-slave-7vjv"
server_key: "f5062da7bd443755dc5d50cfda31eb3a"
server_key_iv: "59a55a49d2560d429cd0e61f3e237eb4"
server_key_version: "encryptionkey@0"
I20260811 07:01:14.976526 9345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "7acb7f418d5e42bb9ce1a43335faaade"
format_stamp: "Formatted at 2026-08-11 07:01:14 on dist-test-slave-7vjv"
server_key: "f5062da7bd443755dc5d50cfda31eb3a"
server_key_iv: "59a55a49d2560d429cd0e61f3e237eb4"
server_key_version: "encryptionkey@0"
I20260811 07:01:14.981684 9345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 07:01:14.985098 9377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:14.986344 9345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:01:14.986538 9345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "7acb7f418d5e42bb9ce1a43335faaade"
format_stamp: "Formatted at 2026-08-11 07:01:14 on dist-test-slave-7vjv"
server_key: "f5062da7bd443755dc5d50cfda31eb3a"
server_key_iv: "59a55a49d2560d429cd0e61f3e237eb4"
server_key_version: "encryptionkey@0"
I20260811 07:01:14.986734 9345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:15.001448 9345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:15.005959 9345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:15.006287 9345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:15.025851 9345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42029
I20260811 07:01:15.025870 9472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42029 every 8 connection(s)
I20260811 07:01:15.027354 9345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:15.028806 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9345
I20260811 07:01:15.029006 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:15.029565 4256 external_mini_cluster.cc:1428] Setting key df2c078d976e1d7ff6777ae5f01bc110
I20260811 07:01:15.031997 9474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 07:01:15 dist-test-slave-7vjv krb5kdc[9319](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:01:15.045629 9474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade: Bootstrap starting.
I20260811 07:01:15.050036 9474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:15.051366 9474 log.cc:824] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:15.054900 9474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade: No bootstrap required, opened a new log
I20260811 07:01:15.055020 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:15.031530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36959 (local address 127.0.0.1:42029)
0811 07:01:15.032398 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:15.032409 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:15.032453 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:01:15.034128 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:15.034148 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:15.034515 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:15.034883 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:15.036879 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:15.038228 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:15.040147 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:15.040816 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:15.040970 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:15.046099 (+ 5129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:15.046109 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:15.046128 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:15.046171 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:15.050352 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:15.051085 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:15.051094 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:15.051103 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:15.051317 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:15.051724 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:15.051729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:15.051732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:15.052458 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:15.052686 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:15.053252 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 07:01:15.053672 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":394,"threads_started":1}
I20260811 07:01:15.058807 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:15.063333 9474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7acb7f418d5e42bb9ce1a43335faaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 07:01:15.063808 9474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:15.063939 9474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7acb7f418d5e42bb9ce1a43335faaade, State: Initialized, Role: FOLLOWER
I20260811 07:01:15.064667 9474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7acb7f418d5e42bb9ce1a43335faaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 07:01:15.064920 9474 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:15.065071 9474 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:15.065260 9474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:15.068065 9474 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7acb7f418d5e42bb9ce1a43335faaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 07:01:15.068697 9474 leader_election.cc:302] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7acb7f418d5e42bb9ce1a43335faaade; no voters:
I20260811 07:01:15.069383 9474 leader_election.cc:288] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:15.069517 9492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:15.070730 9492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [term 1 LEADER]: Becoming Leader. State: Replica: 7acb7f418d5e42bb9ce1a43335faaade, State: Running, Role: LEADER
I20260811 07:01:15.071264 9492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7acb7f418d5e42bb9ce1a43335faaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } }
I20260811 07:01:15.071841 9474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:15.074290 9497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7acb7f418d5e42bb9ce1a43335faaade. Latest consensus state: current_term: 1 leader_uuid: "7acb7f418d5e42bb9ce1a43335faaade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7acb7f418d5e42bb9ce1a43335faaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } } }
I20260811 07:01:15.074390 9495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7acb7f418d5e42bb9ce1a43335faaade" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7acb7f418d5e42bb9ce1a43335faaade" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42029 } } }
I20260811 07:01:15.074738 9497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:15.074822 9495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:15.075757 9500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:15.084071 9500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:15.089003 9477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:15.063385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50543 (local address 127.0.0.1:42029)
0811 07:01:15.063576 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:15.063583 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:15.064021 (+ 438us) server_negotiation.cc:407] Connection header received
0811 07:01:15.064231 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:15.064236 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:15.064317 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:15.064433 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:15.066096 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:15.067232 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:15.068595 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:15.069422 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:15.069569 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:15.073159 (+ 3590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:15.073166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:15.073170 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:15.073213 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:15.078037 (+ 4824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:15.079183 (+ 1146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:15.079188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:15.079194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:15.079441 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:15.080055 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:15.080063 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:15.080073 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:15.081114 (+ 1041us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:15.082684 (+ 1570us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:15.082869 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 07:01:15.083182 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 07:01:15.099720 9500 catalog_manager.cc:1347] Generated new cluster ID: ca5f959424844ad482db50e67fadb77d
I20260811 07:01:15.099835 9500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:15.113799 9500 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:15.115659 9500 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:15.127319 9500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7acb7f418d5e42bb9ce1a43335faaade: Generated new TSK 0
I20260811 07:01:15.128262 9500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:15.133343 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9345
I20260811 07:01:15.134785 9466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:15.223747 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9345
2026-08-11T07:01:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2398 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:15 dist-test-slave-7vjv krb5kdc[9530](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:15 dist-test-slave-7vjv krb5kdc[9530](info): set up 2 sockets
Aug 11 07:01:15 dist-test-slave-7vjv krb5kdc[9530](info): commencing operation
krb5kdc: starting...
W20260811 07:01:16.620576 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:16 dist-test-slave-7vjv krb5kdc[9530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:16.959033 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35039
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:17.302551 9551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:17.302888 9551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:17.303071 9551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:17.310377 9551 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:17.310531 9551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:17.310652 9551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:17.310760 9551 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:17.310846 9551 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:17.320791 9551 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:17.322153 9551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:17.323516 9551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:17.335887 9560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:17.335848 9561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:17.335811 9565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:17.336997 9551 server_base.cc:1034] running on GCE node
I20260811 07:01:17.337793 9551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:17.339097 9551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:17.340279 9551 hybrid_clock.cc:648] HybridClock initialized: now 1786431677340221 us; error 52 us; skew 500 ppm
I20260811 07:01:17.340617 9551 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:17 dist-test-slave-7vjv krb5kdc[9530](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431677, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:17.343745 9551 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:17.345343 9551 webserver.cc:460] Webserver started at http://127.0.0.1:37137/ using document root <none> and password file <none>
I20260811 07:01:17.346055 9551 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:17.346191 9551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:17.346524 9551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:17.349632 9551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "6358b37f947f4de486bb2781429e5964"
format_stamp: "Formatted at 2026-08-11 07:01:17 on dist-test-slave-7vjv"
server_key: "abbadc744e5b6328c16108d1534509c2"
server_key_iv: "38525d8e763d1694b72a85a3182e6045"
server_key_version: "encryptionkey@0"
I20260811 07:01:17.350414 9551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "6358b37f947f4de486bb2781429e5964"
format_stamp: "Formatted at 2026-08-11 07:01:17 on dist-test-slave-7vjv"
server_key: "abbadc744e5b6328c16108d1534509c2"
server_key_iv: "38525d8e763d1694b72a85a3182e6045"
server_key_version: "encryptionkey@0"
I20260811 07:01:17.355648 9551 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 07:01:17.359061 9581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:17.360426 9551 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:17.360611 9551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "6358b37f947f4de486bb2781429e5964"
format_stamp: "Formatted at 2026-08-11 07:01:17 on dist-test-slave-7vjv"
server_key: "abbadc744e5b6328c16108d1534509c2"
server_key_iv: "38525d8e763d1694b72a85a3182e6045"
server_key_version: "encryptionkey@0"
I20260811 07:01:17.360806 9551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:17.383100 9551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:17.387179 9551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:17.387487 9551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:17.406749 9551 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35039
I20260811 07:01:17.406785 9665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35039 every 8 connection(s)
I20260811 07:01:17.408250 9551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:17.410908 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9551
I20260811 07:01:17.411096 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:17.411707 4256 external_mini_cluster.cc:1428] Setting key 8190f65e64714902eb4b22fb796f23e8
I20260811 07:01:17.412693 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:17.425132 9669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964: Bootstrap starting.
I20260811 07:01:17.425488 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:17.413789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54131 (local address 127.0.0.1:35039)
0811 07:01:17.414628 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:17.414640 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:17.414681 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:01:17.416329 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:17.416349 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:17.416722 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:17.417161 (+ 439us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:17.417700 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:17.419062 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:17.420486 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:17.421153 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:17.421353 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:17.422755 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:17.422764 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:17.422781 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:17.423452 (+ 671us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:17.423531 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:17.423559 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:17.423716 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:17.424040 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 07:01:17.424358 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":396,"threads_started":1}
I20260811 07:01:17.428330 9669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:17.429457 9669 log.cc:824] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:17.429613 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:17.432245 9669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964: No bootstrap required, opened a new log
I20260811 07:01:17.441931 9669 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6358b37f947f4de486bb2781429e5964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35039 } }
I20260811 07:01:17.442551 9669 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:17.442682 9669 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6358b37f947f4de486bb2781429e5964, State: Initialized, Role: FOLLOWER
I20260811 07:01:17.443329 9679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:17.434954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40413 (local address 127.0.0.1:35039)
0811 07:01:17.435216 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:17.435227 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:17.436510 (+ 1283us) server_negotiation.cc:407] Connection header received
0811 07:01:17.436692 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:17.436696 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:17.436762 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:17.436904 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:17.437510 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:17.438404 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:17.440370 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:17.440964 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:17.441075 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:17.442508 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:17.442516 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:17.442522 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:17.442555 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:17.442590 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:17.442605 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:17.442722 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:17.442893 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 07:01:17.442984 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 07:01:17.443397 9669 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6358b37f947f4de486bb2781429e5964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35039 } }
I20260811 07:01:17.443727 9669 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:17.443889 9669 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:17.444135 9669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:17.446671 9669 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6358b37f947f4de486bb2781429e5964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35039 } }
I20260811 07:01:17.447206 9669 leader_election.cc:302] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6358b37f947f4de486bb2781429e5964; no voters:
I20260811 07:01:17.447723 9669 leader_election.cc:288] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:17.447856 9693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:17.448637 9693 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [term 1 LEADER]: Becoming Leader. State: Replica: 6358b37f947f4de486bb2781429e5964, State: Running, Role: LEADER
I20260811 07:01:17.449215 9693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6358b37f947f4de486bb2781429e5964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35039 } }
I20260811 07:01:17.449800 9669 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:17.451972 9695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6358b37f947f4de486bb2781429e5964" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6358b37f947f4de486bb2781429e5964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35039 } } }
I20260811 07:01:17.452404 9695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:17.452919 9696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6358b37f947f4de486bb2781429e5964. Latest consensus state: current_term: 1 leader_uuid: "6358b37f947f4de486bb2781429e5964" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6358b37f947f4de486bb2781429e5964" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35039 } } }
I20260811 07:01:17.453222 9696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:17.454916 9711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:17.460875 9711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:17.469413 9711 catalog_manager.cc:1347] Generated new cluster ID: 636f4ca77f2f4ae0825dfabd322d6a49
I20260811 07:01:17.469532 9711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:17.501788 9711 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:17.503126 9711 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:17.517689 9711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6358b37f947f4de486bb2781429e5964: Generated new TSK 0
I20260811 07:01:17.518567 9711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:17.534843 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9551
W20260811 07:01:17.535924 9661 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54131
W20260811 07:01:17.536782 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9551: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:17.536949 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9551
2026-08-11T07:01:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2311 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:17 dist-test-slave-7vjv krb5kdc[9735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:17 dist-test-slave-7vjv krb5kdc[9735](info): set up 2 sockets
Aug 11 07:01:17 dist-test-slave-7vjv krb5kdc[9735](info): commencing operation
krb5kdc: starting...
W20260811 07:01:18.928613 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:19 dist-test-slave-7vjv krb5kdc[9735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431679, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:19.273183 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36813
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:19.615486 9758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:19.615762 9758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:19.615885 9758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:19.622900 9758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:19.623032 9758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:19.623137 9758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:19.623242 9758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:19.623320 9758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:19.632706 9758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:19.633927 9758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:19.635267 9758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:19.647107 9770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:19.647198 9765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:19.647310 9764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:19.648351 9758 server_base.cc:1034] running on GCE node
I20260811 07:01:19.649099 9758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:19.650439 9758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:19.651648 9758 hybrid_clock.cc:648] HybridClock initialized: now 1786431679651615 us; error 30 us; skew 500 ppm
I20260811 07:01:19.652026 9758 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:19 dist-test-slave-7vjv krb5kdc[9735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431679, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:19.659255 9758 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:19.660892 9758 webserver.cc:460] Webserver started at http://127.0.0.1:46845/ using document root <none> and password file <none>
I20260811 07:01:19.661607 9758 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:19.661761 9758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:19.662087 9758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:19.665141 9758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "4c4bff215d8b4138889c54b365b49da9"
format_stamp: "Formatted at 2026-08-11 07:01:19 on dist-test-slave-7vjv"
server_key: "3223cd35e5e9f00eda57e955a6f8fbb3"
server_key_iv: "72ea646b6d7b2b1bf51b9b0b0063146f"
server_key_version: "encryptionkey@0"
I20260811 07:01:19.665905 9758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "4c4bff215d8b4138889c54b365b49da9"
format_stamp: "Formatted at 2026-08-11 07:01:19 on dist-test-slave-7vjv"
server_key: "3223cd35e5e9f00eda57e955a6f8fbb3"
server_key_iv: "72ea646b6d7b2b1bf51b9b0b0063146f"
server_key_version: "encryptionkey@0"
I20260811 07:01:19.671255 9758 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 07:01:19.674674 9782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:19.675876 9758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:01:19.676076 9758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "4c4bff215d8b4138889c54b365b49da9"
format_stamp: "Formatted at 2026-08-11 07:01:19 on dist-test-slave-7vjv"
server_key: "3223cd35e5e9f00eda57e955a6f8fbb3"
server_key_iv: "72ea646b6d7b2b1bf51b9b0b0063146f"
server_key_version: "encryptionkey@0"
I20260811 07:01:19.676339 9758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:19.682114 9758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:19.686190 9758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:19.686506 9758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:19.705724 9758 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36813
I20260811 07:01:19.705821 9870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36813 every 8 connection(s)
I20260811 07:01:19.707142 9758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:19.710863 9873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:19.715515 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9758
I20260811 07:01:19.715703 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:19.716180 4256 external_mini_cluster.cc:1428] Setting key 1809e71fcfc3da24f07dc37f8cd2d199
I20260811 07:01:19.721843 9873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9: Bootstrap starting.
I20260811 07:01:19.723914 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:19.717964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37091 (local address 127.0.0.1:36813)
0811 07:01:19.718792 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:19.718803 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:19.718847 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:01:19.720662 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:19.720684 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:19.721001 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:19.721350 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:19.722408 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:19.722414 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:19.722425 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:19.722440 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:19.722498 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:19.722518 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:19.722750 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:19.722889 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 07:01:19.723055 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":370,"threads_started":1}
I20260811 07:01:19.725121 9873 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:19.726213 9873 log.cc:824] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:19.727573 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:19.729089 9873 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9: No bootstrap required, opened a new log
I20260811 07:01:19.734400 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:19.731928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35185 (local address 127.0.0.1:36813)
0811 07:01:19.732135 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:19.732141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:19.732583 (+ 442us) server_negotiation.cc:407] Connection header received
0811 07:01:19.732700 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:19.732706 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:19.732758 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:19.732870 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:19.733747 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:19.733752 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:19.733757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:19.733773 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:19.733820 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:19.733838 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:19.733952 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:19.734053 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 07:01:19.734116 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:01:19.737358 9873 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4bff215d8b4138889c54b365b49da9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36813 } }
I20260811 07:01:19.737684 9873 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:19.737761 9873 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c4bff215d8b4138889c54b365b49da9, State: Initialized, Role: FOLLOWER
I20260811 07:01:19.738308 9873 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4bff215d8b4138889c54b365b49da9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36813 } }
I20260811 07:01:19.738524 9873 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:19.738618 9873 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:19.738739 9873 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:19.741163 9873 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4bff215d8b4138889c54b365b49da9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36813 } }
I20260811 07:01:19.741593 9873 leader_election.cc:302] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c4bff215d8b4138889c54b365b49da9; no voters:
I20260811 07:01:19.742177 9873 leader_election.cc:288] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:19.742283 9899 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:19.743016 9899 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [term 1 LEADER]: Becoming Leader. State: Replica: 4c4bff215d8b4138889c54b365b49da9, State: Running, Role: LEADER
I20260811 07:01:19.743592 9899 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4bff215d8b4138889c54b365b49da9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36813 } }
I20260811 07:01:19.744328 9873 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:19.747087 9901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c4bff215d8b4138889c54b365b49da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4bff215d8b4138889c54b365b49da9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36813 } } }
I20260811 07:01:19.747453 9901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:19.746639 9902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c4bff215d8b4138889c54b365b49da9. Latest consensus state: current_term: 1 leader_uuid: "4c4bff215d8b4138889c54b365b49da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c4bff215d8b4138889c54b365b49da9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36813 } } }
I20260811 07:01:19.747753 9902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:19.748554 9914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:19.754565 9914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:19.763593 9914 catalog_manager.cc:1347] Generated new cluster ID: 6a88569360944999ba047d0cd20c013b
I20260811 07:01:19.763718 9914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:19.776557 9914 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:19.777036 9914 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:19.784200 9914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c4bff215d8b4138889c54b365b49da9: Generated new TSK 0
I20260811 07:01:19.785143 9914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:19.791548 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9758
W20260811 07:01:19.792949 9861 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37091
W20260811 07:01:19.793637 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9758: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:19.793800 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9758
2026-08-11T07:01:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2255 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:19 dist-test-slave-7vjv krb5kdc[9937](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:19 dist-test-slave-7vjv krb5kdc[9937](info): set up 2 sockets
Aug 11 07:01:19 dist-test-slave-7vjv krb5kdc[9937](info): commencing operation
krb5kdc: starting...
W20260811 07:01:21.201596 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.005s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:21 dist-test-slave-7vjv krb5kdc[9937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:21.539248 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46841
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46841
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:21.887907 9957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:21.888199 9957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:21.888411 9957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:21.895364 9957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:21.895454 9957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:21.895514 9957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:21.895565 9957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:21.895627 9957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:21.905501 9957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46841
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46841
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:21.906824 9957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:21.908264 9957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:21.919764 9964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:21.919739 9969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:21.919849 9967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:21.920888 9957 server_base.cc:1034] running on GCE node
I20260811 07:01:21.921726 9957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:21.923044 9957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:21.924293 9957 hybrid_clock.cc:648] HybridClock initialized: now 1786431681924246 us; error 49 us; skew 500 ppm
I20260811 07:01:21.924645 9957 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:21 dist-test-slave-7vjv krb5kdc[9937](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431681, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:21.927791 9957 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:21.929345 9957 webserver.cc:460] Webserver started at http://127.0.0.1:33793/ using document root <none> and password file <none>
I20260811 07:01:21.929988 9957 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:21.930086 9957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:21.930328 9957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:21.933140 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "472f33ba3e83437892bfcc339077974d"
format_stamp: "Formatted at 2026-08-11 07:01:21 on dist-test-slave-7vjv"
server_key: "348710189b57c9d6310844c32b6b314a"
server_key_iv: "848ebec6302bd42ec9abbed218f18dc8"
server_key_version: "encryptionkey@0"
I20260811 07:01:21.933857 9957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "472f33ba3e83437892bfcc339077974d"
format_stamp: "Formatted at 2026-08-11 07:01:21 on dist-test-slave-7vjv"
server_key: "348710189b57c9d6310844c32b6b314a"
server_key_iv: "848ebec6302bd42ec9abbed218f18dc8"
server_key_version: "encryptionkey@0"
I20260811 07:01:21.938807 9957 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 07:01:21.942178 9980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:21.943512 9957 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 07:01:21.943643 9957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "472f33ba3e83437892bfcc339077974d"
format_stamp: "Formatted at 2026-08-11 07:01:21 on dist-test-slave-7vjv"
server_key: "348710189b57c9d6310844c32b6b314a"
server_key_iv: "848ebec6302bd42ec9abbed218f18dc8"
server_key_version: "encryptionkey@0"
I20260811 07:01:21.943800 9957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:21.960393 9957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:21.964802 9957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:21.965039 9957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:21.984136 9957 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46841
I20260811 07:01:21.984196 10061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46841 every 8 connection(s)
I20260811 07:01:21.985587 9957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:21.989396 10064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:21.994081 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 9957
I20260811 07:01:21.994285 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:21.994897 4256 external_mini_cluster.cc:1428] Setting key 1ead3a32b17de3fc1b226ee901411b60
I20260811 07:01:22.002583 10064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d: Bootstrap starting.
I20260811 07:01:22.006557 10064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:22.007915 10064 log.cc:824] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:22.010886 10064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d: No bootstrap required, opened a new log
Aug 11 07:01:22 dist-test-slave-7vjv krb5kdc[9937](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431681, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:01:22.019825 10064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472f33ba3e83437892bfcc339077974d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46841 } }
I20260811 07:01:22.020255 10064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:22.020381 10064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 472f33ba3e83437892bfcc339077974d, State: Initialized, Role: FOLLOWER
I20260811 07:01:22.021075 10064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472f33ba3e83437892bfcc339077974d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46841 } }
I20260811 07:01:22.021359 10064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:22.021522 10064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:22.021703 10064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:22.022584 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:21.997140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:38535 (local address 127.0.0.1:46841)
0811 07:01:21.998196 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:21.998212 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:21.998563 (+ 351us) server_negotiation.cc:407] Connection header received
0811 07:01:22.001282 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:22.001308 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:22.001805 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:22.002312 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:22.003675 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:22.005817 (+ 2142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:22.007288 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:22.008449 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:22.008691 (+ 242us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:22.013757 (+ 5066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:22.013767 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:22.013781 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:22.013859 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:22.017622 (+ 3763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:22.018649 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:22.018656 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:22.018661 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:22.018878 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:22.019401 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:22.019406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:22.019407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:22.019993 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:22.020391 (+ 398us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:22.020919 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 07:01:22.021282 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":457,"threads_started":1}
I20260811 07:01:22.024374 10064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472f33ba3e83437892bfcc339077974d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46841 } }
I20260811 07:01:22.024919 10064 leader_election.cc:302] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 472f33ba3e83437892bfcc339077974d; no voters:
I20260811 07:01:22.025633 10064 leader_election.cc:288] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:22.025764 10069 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:22.026850 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:22.027072 10069 raft_consensus.cc:686] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [term 1 LEADER]: Becoming Leader. State: Replica: 472f33ba3e83437892bfcc339077974d, State: Running, Role: LEADER
I20260811 07:01:22.027613 10069 consensus_queue.cc:237] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472f33ba3e83437892bfcc339077974d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46841 } }
I20260811 07:01:22.027736 10064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:22.030293 10072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "472f33ba3e83437892bfcc339077974d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472f33ba3e83437892bfcc339077974d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46841 } } }
I20260811 07:01:22.030692 10072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:22.031184 10075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 472f33ba3e83437892bfcc339077974d. Latest consensus state: current_term: 1 leader_uuid: "472f33ba3e83437892bfcc339077974d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "472f33ba3e83437892bfcc339077974d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46841 } } }
I20260811 07:01:22.031486 10075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:22.035481 10081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:22.047245 10081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:22.062255 10081 catalog_manager.cc:1347] Generated new cluster ID: e4e27c2df1554876a38bd8c545a79f1b
I20260811 07:01:22.062377 10081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:22.066327 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:22.038369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:49433 (local address 127.0.0.1:46841)
0811 07:01:22.038594 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:22.038603 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:22.048212 (+ 9609us) server_negotiation.cc:407] Connection header received
0811 07:01:22.048518 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:22.048529 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:22.048612 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:22.048690 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:22.050870 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:22.052155 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:22.054066 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:22.054786 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:22.054925 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:22.058864 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:22.058871 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:22.058878 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:22.058916 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:22.062709 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:22.063762 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:22.063767 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:22.063771 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:22.063934 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:22.064564 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:22.064570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:22.064572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:22.064952 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:22.065166 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:22.065807 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 07:01:22.065983 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 07:01:22.102013 10081 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:22.103413 10081 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:22.109323 10081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 472f33ba3e83437892bfcc339077974d: Generated new TSK 0
I20260811 07:01:22.110106 10081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:22.118714 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 9957
I20260811 07:01:22.120019 10053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:22.213578 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 9957
W20260811 07:01:22.242038 9932 connection.cc:535] client connection to 127.0.0.1:46841 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46841): Connection reset by peer (error 104)
2026-08-11T07:01:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2421 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:22 dist-test-slave-7vjv krb5kdc[10116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:22 dist-test-slave-7vjv krb5kdc[10116](info): set up 2 sockets
Aug 11 07:01:22 dist-test-slave-7vjv krb5kdc[10116](info): commencing operation
krb5kdc: starting...
W20260811 07:01:23.681665 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.420s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:23 dist-test-slave-7vjv krb5kdc[10116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431683, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:24.020684 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42739
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:24.369719 10142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:24.369997 10142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:24.370170 10142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:24.377290 10142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:24.377417 10142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:24.377532 10142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:24.377645 10142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:24.377722 10142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:24.387389 10142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:24.388713 10142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:24.390031 10142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:24.401734 10157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:24.401814 10150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:24.401790 10155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:24.402778 10142 server_base.cc:1034] running on GCE node
I20260811 07:01:24.403560 10142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:24.404959 10142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:24.406205 10142 hybrid_clock.cc:648] HybridClock initialized: now 1786431684406157 us; error 52 us; skew 500 ppm
I20260811 07:01:24.406558 10142 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:24 dist-test-slave-7vjv krb5kdc[10116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431684, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:24.409760 10142 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:24.411193 10142 webserver.cc:460] Webserver started at http://127.0.0.1:35955/ using document root <none> and password file <none>
I20260811 07:01:24.411890 10142 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:24.412042 10142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:24.412374 10142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:24.415310 10142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "860bcdea6c0346f3a0bb5fbcf60d683c"
format_stamp: "Formatted at 2026-08-11 07:01:24 on dist-test-slave-7vjv"
server_key: "2c76ab13d8f07da1636b44e82e7d109e"
server_key_iv: "27523c026eba62e2ea0fb2bb9b22cddd"
server_key_version: "encryptionkey@0"
I20260811 07:01:24.416239 10142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "860bcdea6c0346f3a0bb5fbcf60d683c"
format_stamp: "Formatted at 2026-08-11 07:01:24 on dist-test-slave-7vjv"
server_key: "2c76ab13d8f07da1636b44e82e7d109e"
server_key_iv: "27523c026eba62e2ea0fb2bb9b22cddd"
server_key_version: "encryptionkey@0"
I20260811 07:01:24.421370 10142 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 07:01:24.424765 10165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:24.425989 10142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 07:01:24.426191 10142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "860bcdea6c0346f3a0bb5fbcf60d683c"
format_stamp: "Formatted at 2026-08-11 07:01:24 on dist-test-slave-7vjv"
server_key: "2c76ab13d8f07da1636b44e82e7d109e"
server_key_iv: "27523c026eba62e2ea0fb2bb9b22cddd"
server_key_version: "encryptionkey@0"
I20260811 07:01:24.426397 10142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:24.465001 10142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:24.468868 10142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:24.469161 10142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:24.488883 10142 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42739
I20260811 07:01:24.488956 10255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42739 every 8 connection(s)
I20260811 07:01:24.491330 10142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:24.493340 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10142
I20260811 07:01:24.493533 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:24.494144 4256 external_mini_cluster.cc:1428] Setting key 065c8139f2da578b49416ec204573ab4
I20260811 07:01:24.497609 10256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:24.510762 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:24.495838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:43203 (local address 127.0.0.1:42739)
0811 07:01:24.496978 (+ 1140us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:24.496990 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:24.497028 (+ 38us) server_negotiation.cc:407] Connection header received
0811 07:01:24.499250 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:24.499271 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:24.499653 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:24.500015 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:24.500648 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:24.503045 (+ 2397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:24.505024 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:24.506068 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:24.506231 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:24.507884 (+ 1653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:24.507892 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:24.507905 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:24.508603 (+ 698us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:24.508678 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:24.508703 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:24.508869 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:24.509161 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 07:01:24.509337 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":935,"thread_start_us":663,"threads_started":1}
I20260811 07:01:24.514991 10256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c: Bootstrap starting.
I20260811 07:01:24.515146 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:24.519201 10256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:24.520567 10256 log.cc:824] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:24.525455 10256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c: No bootstrap required, opened a new log
I20260811 07:01:24.537329 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:24.524321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:35097 (local address 127.0.0.1:42739)
0811 07:01:24.524510 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:24.524514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:24.529158 (+ 4644us) server_negotiation.cc:407] Connection header received
0811 07:01:24.529395 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:24.529400 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:24.529505 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:24.529617 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:24.530268 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:24.531642 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:24.533604 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:24.534443 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:24.534598 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:24.536010 (+ 1412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:24.536020 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:24.536026 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:24.536052 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:24.536157 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:24.536187 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:24.536334 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:24.536824 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 07:01:24.536892 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:01:24.537585 10256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260811 07:01:24.538018 10256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:24.538148 10256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 860bcdea6c0346f3a0bb5fbcf60d683c, State: Initialized, Role: FOLLOWER
I20260811 07:01:24.538853 10256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260811 07:01:24.539150 10256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:24.539296 10256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:24.539474 10256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:24.541778 10256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260811 07:01:24.542276 10256 leader_election.cc:302] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 860bcdea6c0346f3a0bb5fbcf60d683c; no voters:
I20260811 07:01:24.542925 10256 leader_election.cc:288] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:24.543141 10275 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:24.544668 10275 raft_consensus.cc:686] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [term 1 LEADER]: Becoming Leader. State: Replica: 860bcdea6c0346f3a0bb5fbcf60d683c, State: Running, Role: LEADER
I20260811 07:01:24.545426 10256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:24.545329 10275 consensus_queue.cc:237] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } }
I20260811 07:01:24.548400 10277 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 860bcdea6c0346f3a0bb5fbcf60d683c. Latest consensus state: current_term: 1 leader_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } } }
I20260811 07:01:24.548789 10277 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:24.549299 10276 sys_catalog.cc:455] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "860bcdea6c0346f3a0bb5fbcf60d683c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42739 } } }
I20260811 07:01:24.549667 10276 sys_catalog.cc:458] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:24.549974 10286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:24.557868 10286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:24.567390 10286 catalog_manager.cc:1347] Generated new cluster ID: 59f489de08c74f3a8e74052c06bcf527
I20260811 07:01:24.567502 10286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:24.590982 10286 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:24.592747 10286 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:24.599007 10286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 860bcdea6c0346f3a0bb5fbcf60d683c: Generated new TSK 0
I20260811 07:01:24.599831 10286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:24.620389 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10142
W20260811 07:01:24.621663 10250 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:43203
W20260811 07:01:24.622563 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10142: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:24.622733 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10142
W20260811 07:01:24.649631 10112 connection.cc:535] client connection to 127.0.0.1:42739 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42739): Connection reset by peer (error 104)
2026-08-11T07:01:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:24 dist-test-slave-7vjv krb5kdc[10307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:24 dist-test-slave-7vjv krb5kdc[10307](info): set up 2 sockets
Aug 11 07:01:24 dist-test-slave-7vjv krb5kdc[10307](info): commencing operation
krb5kdc: starting...
W20260811 07:01:26.022346 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.353s user 0.002s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:26 dist-test-slave-7vjv krb5kdc[10307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:26.366779 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:26.712452 10327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:26.712715 10327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:26.712886 10327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:26.719993 10327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:26.720131 10327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:26.720235 10327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:26.720338 10327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:26.720417 10327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:26.730240 10327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:26.731457 10327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:26.732795 10327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:26.744912 10337 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:26.744949 10338 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:26.745004 10340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:26.745807 10327 server_base.cc:1034] running on GCE node
I20260811 07:01:26.746565 10327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:26.747975 10327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:26.749233 10327 hybrid_clock.cc:648] HybridClock initialized: now 1786431686749201 us; error 58 us; skew 500 ppm
I20260811 07:01:26.749576 10327 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:26 dist-test-slave-7vjv krb5kdc[10307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431686, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:26.752898 10327 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:26.754360 10327 webserver.cc:460] Webserver started at http://127.0.0.1:44055/ using document root <none> and password file <none>
I20260811 07:01:26.755025 10327 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:26.755168 10327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:26.755471 10327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:26.758507 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "f985e398375d4d84975d90154cd29abd"
format_stamp: "Formatted at 2026-08-11 07:01:26 on dist-test-slave-7vjv"
server_key: "aac2d1835451340b41f3c1d4479d3334"
server_key_iv: "05784e628fd54d1abcec047d20879b0e"
server_key_version: "encryptionkey@0"
I20260811 07:01:26.759274 10327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "f985e398375d4d84975d90154cd29abd"
format_stamp: "Formatted at 2026-08-11 07:01:26 on dist-test-slave-7vjv"
server_key: "aac2d1835451340b41f3c1d4479d3334"
server_key_iv: "05784e628fd54d1abcec047d20879b0e"
server_key_version: "encryptionkey@0"
I20260811 07:01:26.764318 10327 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260811 07:01:26.767618 10358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:26.768962 10327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:26.769148 10327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "f985e398375d4d84975d90154cd29abd"
format_stamp: "Formatted at 2026-08-11 07:01:26 on dist-test-slave-7vjv"
server_key: "aac2d1835451340b41f3c1d4479d3334"
server_key_iv: "05784e628fd54d1abcec047d20879b0e"
server_key_version: "encryptionkey@0"
I20260811 07:01:26.769343 10327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:26.774806 10327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:26.779105 10327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:26.779426 10327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:26.797734 10327 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43859
I20260811 07:01:26.797788 10459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43859 every 8 connection(s)
I20260811 07:01:26.799081 10327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:26.800354 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10327
I20260811 07:01:26.800557 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:26.801100 4256 external_mini_cluster.cc:1428] Setting key 80e8fba97e7b1e216bd9ebfe6db7191e
I20260811 07:01:26.803628 10462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:26.808903 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:26.802932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:42453 (local address 127.0.0.1:43859)
0811 07:01:26.803821 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:26.803833 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:26.803874 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:01:26.805530 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:26.805550 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:26.805867 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:26.806203 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:26.807275 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:26.807281 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:26.807293 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:26.807307 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:26.807387 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:26.807410 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:26.807577 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:26.807720 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 07:01:26.807934 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":382,"threads_started":1}
I20260811 07:01:26.812479 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:26.816311 10462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd: Bootstrap starting.
I20260811 07:01:26.820546 10462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:26.820662 10467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:26.817998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:43923 (local address 127.0.0.1:43859)
0811 07:01:26.818221 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:26.818226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:26.818559 (+ 333us) server_negotiation.cc:407] Connection header received
0811 07:01:26.818728 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:26.818731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:26.818777 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:26.818854 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:26.819943 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:26.819949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:26.819953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:26.819972 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:26.820018 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:26.820033 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:26.820176 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:26.820273 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 07:01:26.820334 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 07:01:26.821924 10462 log.cc:824] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:26.824635 10462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd: No bootstrap required, opened a new log
I20260811 07:01:26.832293 10462 raft_consensus.cc:348] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f985e398375d4d84975d90154cd29abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 07:01:26.832628 10462 raft_consensus.cc:374] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:26.832753 10462 raft_consensus.cc:729] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f985e398375d4d84975d90154cd29abd, State: Initialized, Role: FOLLOWER
I20260811 07:01:26.833484 10462 consensus_queue.cc:260] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f985e398375d4d84975d90154cd29abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 07:01:26.833789 10462 raft_consensus.cc:388] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:26.833907 10462 raft_consensus.cc:482] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:26.834043 10462 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:26.836191 10462 raft_consensus.cc:504] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f985e398375d4d84975d90154cd29abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 07:01:26.836621 10462 leader_election.cc:302] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f985e398375d4d84975d90154cd29abd; no voters:
I20260811 07:01:26.837242 10462 leader_election.cc:288] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:26.837371 10479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:26.838274 10479 raft_consensus.cc:686] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [term 1 LEADER]: Becoming Leader. State: Replica: f985e398375d4d84975d90154cd29abd, State: Running, Role: LEADER
I20260811 07:01:26.838831 10479 consensus_queue.cc:237] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f985e398375d4d84975d90154cd29abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } }
I20260811 07:01:26.839383 10462 sys_catalog.cc:564] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:26.841854 10480 sys_catalog.cc:455] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f985e398375d4d84975d90154cd29abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f985e398375d4d84975d90154cd29abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260811 07:01:26.842353 10480 sys_catalog.cc:458] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:26.841987 10482 sys_catalog.cc:455] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f985e398375d4d84975d90154cd29abd. Latest consensus state: current_term: 1 leader_uuid: "f985e398375d4d84975d90154cd29abd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f985e398375d4d84975d90154cd29abd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43859 } } }
I20260811 07:01:26.842715 10482 sys_catalog.cc:458] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:26.843828 10489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:26.849141 10489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:26.857741 10489 catalog_manager.cc:1347] Generated new cluster ID: a79a9f5fad8646c6ab3a08352b54d189
I20260811 07:01:26.857882 10489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:26.870478 10489 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:26.870966 10489 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:26.879251 10489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f985e398375d4d84975d90154cd29abd: Generated new TSK 0
I20260811 07:01:26.880038 10489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:26.895859 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10327
W20260811 07:01:26.896986 10449 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:42453
W20260811 07:01:26.897789 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10327: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:26.897946 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10327
2026-08-11T07:01:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:27 dist-test-slave-7vjv krb5kdc[10512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:27 dist-test-slave-7vjv krb5kdc[10512](info): set up 2 sockets
Aug 11 07:01:27 dist-test-slave-7vjv krb5kdc[10512](info): commencing operation
krb5kdc: starting...
W20260811 07:01:28.293992 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.347s user 0.003s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:28 dist-test-slave-7vjv krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:28.649279 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:28.994583 10534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:28.994853 10534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:28.995044 10534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:29.002045 10534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:29.002144 10534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:29.002190 10534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:29.002302 10534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:29.002398 10534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:29.012077 10534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:29.013450 10534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:29.014859 10534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:29.026702 10546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:29.026923 10544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:29.026993 10548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:29.027968 10534 server_base.cc:1034] running on GCE node
I20260811 07:01:29.028777 10534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:29.030149 10534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:29.031381 10534 hybrid_clock.cc:648] HybridClock initialized: now 1786431689031340 us; error 50 us; skew 500 ppm
I20260811 07:01:29.031729 10534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:29 dist-test-slave-7vjv krb5kdc[10512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431689, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:29.034912 10534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:29.036598 10534 webserver.cc:460] Webserver started at http://127.0.0.1:40721/ using document root <none> and password file <none>
I20260811 07:01:29.037315 10534 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:29.037463 10534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:29.037774 10534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:29.040951 10534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "298e40a8266444c78e751fd2e2cc210b"
format_stamp: "Formatted at 2026-08-11 07:01:29 on dist-test-slave-7vjv"
server_key: "fc898e266ee8c54f8d4a6bd029292382"
server_key_iv: "59c23aae9de16c641e4910c0f17b7a8f"
server_key_version: "encryptionkey@0"
I20260811 07:01:29.041738 10534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "298e40a8266444c78e751fd2e2cc210b"
format_stamp: "Formatted at 2026-08-11 07:01:29 on dist-test-slave-7vjv"
server_key: "fc898e266ee8c54f8d4a6bd029292382"
server_key_iv: "59c23aae9de16c641e4910c0f17b7a8f"
server_key_version: "encryptionkey@0"
I20260811 07:01:29.046957 10534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 07:01:29.050390 10565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:29.051545 10534 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 07:01:29.051723 10534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "298e40a8266444c78e751fd2e2cc210b"
format_stamp: "Formatted at 2026-08-11 07:01:29 on dist-test-slave-7vjv"
server_key: "fc898e266ee8c54f8d4a6bd029292382"
server_key_iv: "59c23aae9de16c641e4910c0f17b7a8f"
server_key_version: "encryptionkey@0"
I20260811 07:01:29.051913 10534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:29.075402 10534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:29.079396 10534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:29.079720 10534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:29.098353 10534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40975
I20260811 07:01:29.098438 10657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40975 every 8 connection(s)
I20260811 07:01:29.099772 10534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:29.102497 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10534
I20260811 07:01:29.102703 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:29.103267 4256 external_mini_cluster.cc:1428] Setting key d6a3a40c44c2ef65a76041fa030309a8
I20260811 07:01:29.103881 10658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:29.116456 10658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b: Bootstrap starting.
Aug 11 07:01:29 dist-test-slave-7vjv krb5kdc[10512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431688, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 07:01:29.120304 10658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:29.121397 10658 log.cc:824] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:29.124742 10658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b: No bootstrap required, opened a new log
I20260811 07:01:29.125903 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:29.105258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:56165 (local address 127.0.0.1:40975)
0811 07:01:29.106021 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:29.106033 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:29.106074 (+ 41us) server_negotiation.cc:407] Connection header received
0811 07:01:29.107596 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:29.107616 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:29.108019 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:29.108493 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:29.110014 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:29.111317 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:29.112738 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:29.113425 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:29.113659 (+ 234us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:29.118460 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:29.118468 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:29.118480 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:29.118520 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:29.121643 (+ 3123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:29.122548 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:29.122556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:29.122565 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:29.122795 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:29.123277 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:29.123281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:29.123283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:29.123760 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:29.124016 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:29.124596 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 07:01:29.124891 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":333,"threads_started":1}
I20260811 07:01:29.129827 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:29.134411 10658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298e40a8266444c78e751fd2e2cc210b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40975 } }
I20260811 07:01:29.134864 10658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:29.135025 10658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 298e40a8266444c78e751fd2e2cc210b, State: Initialized, Role: FOLLOWER
I20260811 07:01:29.135737 10658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298e40a8266444c78e751fd2e2cc210b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40975 } }
I20260811 07:01:29.136054 10658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:29.136255 10658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:29.136442 10658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:29.138715 10658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298e40a8266444c78e751fd2e2cc210b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40975 } }
I20260811 07:01:29.139212 10658 leader_election.cc:302] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 298e40a8266444c78e751fd2e2cc210b; no voters:
I20260811 07:01:29.139813 10658 leader_election.cc:288] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:29.139962 10680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:29.141129 10680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [term 1 LEADER]: Becoming Leader. State: Replica: 298e40a8266444c78e751fd2e2cc210b, State: Running, Role: LEADER
I20260811 07:01:29.141808 10680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298e40a8266444c78e751fd2e2cc210b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40975 } }
I20260811 07:01:29.142236 10658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:29.144865 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "298e40a8266444c78e751fd2e2cc210b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298e40a8266444c78e751fd2e2cc210b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40975 } } }
I20260811 07:01:29.145308 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:29.144892 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 298e40a8266444c78e751fd2e2cc210b. Latest consensus state: current_term: 1 leader_uuid: "298e40a8266444c78e751fd2e2cc210b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "298e40a8266444c78e751fd2e2cc210b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40975 } } }
I20260811 07:01:29.145465 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:29.146258 10693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:29.155241 10667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:29.135209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:34103 (local address 127.0.0.1:40975)
0811 07:01:29.135427 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:29.135436 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:29.136043 (+ 607us) server_negotiation.cc:407] Connection header received
0811 07:01:29.136240 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:29.136246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:29.136317 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:29.136447 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:29.137872 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:29.138672 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:29.140390 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:29.141311 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:29.141410 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:29.145400 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:29.145409 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:29.145417 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:29.145486 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:29.150342 (+ 4856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:29.151628 (+ 1286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:29.151637 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:29.151644 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:29.151860 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:29.152617 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:29.152625 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:29.152631 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:29.153108 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:29.153453 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:29.154407 (+ 954us) server_negotiation.cc:299] Negotiation successful
0811 07:01:29.154683 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:01:29.159204 10693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:29.169513 10693 catalog_manager.cc:1347] Generated new cluster ID: ffc2b304025e4e928f1a3bb2e62e4f3f
I20260811 07:01:29.169636 10693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:29.178419 10693 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:29.180261 10693 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:29.190001 10693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 298e40a8266444c78e751fd2e2cc210b: Generated new TSK 0
I20260811 07:01:29.190793 10693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:29.200848 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10534
I20260811 07:01:29.201892 10650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:29.292613 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10534
W20260811 07:01:29.317390 10510 connection.cc:535] client connection to 127.0.0.1:40975 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40975): Connection reset by peer (error 104)
2026-08-11T07:01:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:29 dist-test-slave-7vjv krb5kdc[10721](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:29 dist-test-slave-7vjv krb5kdc[10721](info): set up 2 sockets
Aug 11 07:01:29 dist-test-slave-7vjv krb5kdc[10721](info): commencing operation
krb5kdc: starting...
W20260811 07:01:30.716427 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.378s user 0.003s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:30 dist-test-slave-7vjv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:31.061426 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:31.419319 10751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:31.419687 10751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:31.419875 10751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:31.427136 10751 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:31.427304 10751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:31.427423 10751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:31.427541 10751 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:31.427625 10751 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:31.437672 10751 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35803
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:31.439087 10751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:31.440586 10751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:31.452821 10757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:31.452811 10758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:31.452872 10766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:31.453789 10751 server_base.cc:1034] running on GCE node
I20260811 07:01:31.454630 10751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:31.456146 10751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:31.457522 10751 hybrid_clock.cc:648] HybridClock initialized: now 1786431691457429 us; error 75 us; skew 500 ppm
I20260811 07:01:31.457906 10751 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:31 dist-test-slave-7vjv krb5kdc[10721](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431691, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:31.465619 10751 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:31.467232 10751 webserver.cc:460] Webserver started at http://127.0.0.1:36571/ using document root <none> and password file <none>
I20260811 07:01:31.467908 10751 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:31.468025 10751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:31.468451 10751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:31.471657 10751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "d8ba9476929949169fc8a7669e7aceb7"
format_stamp: "Formatted at 2026-08-11 07:01:31 on dist-test-slave-7vjv"
server_key: "24611465922e8da8b7e008a25fc1bb40"
server_key_iv: "f0fd9711495be9418d89040f72871851"
server_key_version: "encryptionkey@0"
I20260811 07:01:31.472535 10751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "d8ba9476929949169fc8a7669e7aceb7"
format_stamp: "Formatted at 2026-08-11 07:01:31 on dist-test-slave-7vjv"
server_key: "24611465922e8da8b7e008a25fc1bb40"
server_key_iv: "f0fd9711495be9418d89040f72871851"
server_key_version: "encryptionkey@0"
I20260811 07:01:31.477906 10751 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 07:01:31.481511 10776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:31.482823 10751 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:01:31.482975 10751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "d8ba9476929949169fc8a7669e7aceb7"
format_stamp: "Formatted at 2026-08-11 07:01:31 on dist-test-slave-7vjv"
server_key: "24611465922e8da8b7e008a25fc1bb40"
server_key_iv: "f0fd9711495be9418d89040f72871851"
server_key_version: "encryptionkey@0"
I20260811 07:01:31.483151 10751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:31.505124 10751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:31.509704 10751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:31.509994 10751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:31.530431 10751 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35803
I20260811 07:01:31.530453 10866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35803 every 8 connection(s)
I20260811 07:01:31.531971 10751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:31.535362 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10751
I20260811 07:01:31.535583 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:31.535693 10871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:31.536288 4256 external_mini_cluster.cc:1428] Setting key 0e4b3e4fb804a7829dca228875eb916a
I20260811 07:01:31.549810 10871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7: Bootstrap starting.
I20260811 07:01:31.550602 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:31.538313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:54139 (local address 127.0.0.1:35803)
0811 07:01:31.539293 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:31.539304 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:31.539356 (+ 52us) server_negotiation.cc:407] Connection header received
0811 07:01:31.541058 (+ 1702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:31.541083 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:31.541487 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:31.541806 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:31.542454 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:31.543856 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:31.545854 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:31.546613 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:31.546760 (+ 147us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:31.548202 (+ 1442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:31.548212 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:31.548225 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:31.548790 (+ 565us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:31.548874 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:31.548901 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:31.549038 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:31.549361 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 07:01:31.549558 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":494,"threads_started":1}
I20260811 07:01:31.552903 10871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:31.554000 10871 log.cc:824] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:31.554587 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:31.557395 10871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7: No bootstrap required, opened a new log
I20260811 07:01:31.567049 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:31.560270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:52873 (local address 127.0.0.1:35803)
0811 07:01:31.560458 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:31.560465 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:31.560753 (+ 288us) server_negotiation.cc:407] Connection header received
0811 07:01:31.560912 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:31.560923 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:31.561008 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:31.561156 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:31.561656 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:31.562806 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:31.564151 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:31.564905 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:31.565096 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:31.566163 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:31.566171 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:31.566177 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:31.566199 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:31.566270 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:31.566302 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:31.566446 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:31.566612 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 07:01:31.566676 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 07:01:31.568029 10871 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ba9476929949169fc8a7669e7aceb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35803 } }
I20260811 07:01:31.568629 10871 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:31.568742 10871 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8ba9476929949169fc8a7669e7aceb7, State: Initialized, Role: FOLLOWER
I20260811 07:01:31.569351 10871 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ba9476929949169fc8a7669e7aceb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35803 } }
I20260811 07:01:31.569628 10871 raft_consensus.cc:388] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:31.569797 10871 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:31.569983 10871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:31.572301 10871 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ba9476929949169fc8a7669e7aceb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35803 } }
I20260811 07:01:31.572885 10871 leader_election.cc:302] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8ba9476929949169fc8a7669e7aceb7; no voters:
I20260811 07:01:31.573616 10871 leader_election.cc:288] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:31.573731 10886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:31.574546 10886 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [term 1 LEADER]: Becoming Leader. State: Replica: d8ba9476929949169fc8a7669e7aceb7, State: Running, Role: LEADER
I20260811 07:01:31.575137 10886 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ba9476929949169fc8a7669e7aceb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35803 } }
I20260811 07:01:31.575908 10871 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:31.578102 10889 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8ba9476929949169fc8a7669e7aceb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ba9476929949169fc8a7669e7aceb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35803 } } }
I20260811 07:01:31.578537 10889 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:31.583866 10890 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8ba9476929949169fc8a7669e7aceb7. Latest consensus state: current_term: 1 leader_uuid: "d8ba9476929949169fc8a7669e7aceb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8ba9476929949169fc8a7669e7aceb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35803 } } }
I20260811 07:01:31.584357 10890 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:31.585142 10899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:31.592025 10899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:31.601830 10899 catalog_manager.cc:1347] Generated new cluster ID: 36ca5ec2ff0848ad92c49514fbb8a23c
I20260811 07:01:31.601966 10899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:31.612634 10899 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:31.614416 10899 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:31.630551 10899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8ba9476929949169fc8a7669e7aceb7: Generated new TSK 0
I20260811 07:01:31.631569 10899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:31.637459 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10751
W20260811 07:01:31.638736 10859 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:54139
W20260811 07:01:31.639605 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10751: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:31.639778 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10751
W20260811 07:01:31.668365 10715 connection.cc:535] client connection to 127.0.0.1:35803 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35803): Connection reset by peer (error 104)
2026-08-11T07:01:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2349 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:31 dist-test-slave-7vjv krb5kdc[10929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:31 dist-test-slave-7vjv krb5kdc[10929](info): set up 2 sockets
Aug 11 07:01:31 dist-test-slave-7vjv krb5kdc[10929](info): commencing operation
krb5kdc: starting...
W20260811 07:01:33.038452 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.003s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:33 dist-test-slave-7vjv krb5kdc[10929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 07:01:33.381268 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38847
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39415
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:33.729271 10955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:33.729542 10955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:33.729738 10955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:33.736804 10955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:33.736908 10955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:33.737033 10955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:33.737141 10955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:33.737227 10955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:33.746910 10955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39415
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38847
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:33.748180 10955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:33.749480 10955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:33.761507 10960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:33.761526 10961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:33.761641 10963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:33.762326 10955 server_base.cc:1034] running on GCE node
I20260811 07:01:33.763077 10955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:33.764521 10955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:33.765759 10955 hybrid_clock.cc:648] HybridClock initialized: now 1786431693765730 us; error 42 us; skew 500 ppm
I20260811 07:01:33.766121 10955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:33 dist-test-slave-7vjv krb5kdc[10929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431693, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:33.769299 10955 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 07:01:33.770699 10955 webserver.cc:460] Webserver started at http://127.0.0.1:40565/ using document root <none> and password file <none>
I20260811 07:01:33.771400 10955 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:33.771557 10955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:33.771909 10955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:33.774853 10955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "78369c5e3f6f4dbfa1fa5d1f511de286"
format_stamp: "Formatted at 2026-08-11 07:01:33 on dist-test-slave-7vjv"
server_key: "4e7b9352afef33a4b74c7707c0e19320"
server_key_iv: "3e602e99e51dd95d808070e019689a9c"
server_key_version: "encryptionkey@0"
I20260811 07:01:33.775677 10955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "78369c5e3f6f4dbfa1fa5d1f511de286"
format_stamp: "Formatted at 2026-08-11 07:01:33 on dist-test-slave-7vjv"
server_key: "4e7b9352afef33a4b74c7707c0e19320"
server_key_iv: "3e602e99e51dd95d808070e019689a9c"
server_key_version: "encryptionkey@0"
I20260811 07:01:33.780803 10955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 07:01:33.784394 10976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:33.785624 10955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 07:01:33.785806 10955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "78369c5e3f6f4dbfa1fa5d1f511de286"
format_stamp: "Formatted at 2026-08-11 07:01:33 on dist-test-slave-7vjv"
server_key: "4e7b9352afef33a4b74c7707c0e19320"
server_key_iv: "3e602e99e51dd95d808070e019689a9c"
server_key_version: "encryptionkey@0"
I20260811 07:01:33.786002 10955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:33.791620 10955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:33.795265 10955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:33.795565 10955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:33.814036 10955 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38847
I20260811 07:01:33.814116 11070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38847 every 8 connection(s)
I20260811 07:01:33.815421 10955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:33.819087 11071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:33.822985 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 10955
I20260811 07:01:33.823176 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:33.823630 4256 external_mini_cluster.cc:1428] Setting key 6451b97885c5198e9d665d2deacbb90a
I20260811 07:01:33.830850 11071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286: Bootstrap starting.
I20260811 07:01:33.832535 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:33.825498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:46625 (local address 127.0.0.1:38847)
0811 07:01:33.826327 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:33.826338 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:33.826380 (+ 42us) server_negotiation.cc:407] Connection header received
0811 07:01:33.829249 (+ 2869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:33.829272 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:33.829577 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:33.829879 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:33.831040 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:33.831046 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:33.831057 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:33.831072 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:33.831131 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:33.831154 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:33.831315 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:33.831476 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 07:01:33.831616 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":376,"threads_started":1}
I20260811 07:01:33.834306 11071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:33.835590 11071 log.cc:824] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:33.836081 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:33.838281 11071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286: No bootstrap required, opened a new log
I20260811 07:01:33.843338 11076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:33.840502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.91:48833 (local address 127.0.0.1:38847)
0811 07:01:33.840739 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:33.840748 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:33.841354 (+ 606us) server_negotiation.cc:407] Connection header received
0811 07:01:33.841549 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:33.841563 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:33.841613 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:33.841730 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:33.842653 (+ 923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:33.842662 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:33.842666 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:33.842684 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:33.842733 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:33.842750 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:33.842882 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:33.842979 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 07:01:33.843043 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:01:33.846561 11071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38847 } }
I20260811 07:01:33.846946 11071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:33.847062 11071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78369c5e3f6f4dbfa1fa5d1f511de286, State: Initialized, Role: FOLLOWER
I20260811 07:01:33.847595 11071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38847 } }
I20260811 07:01:33.847857 11071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:33.848042 11071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:33.848261 11071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:33.850836 11071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38847 } }
I20260811 07:01:33.851346 11071 leader_election.cc:302] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78369c5e3f6f4dbfa1fa5d1f511de286; no voters:
I20260811 07:01:33.851915 11071 leader_election.cc:288] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:33.852065 11089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:33.852880 11089 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [term 1 LEADER]: Becoming Leader. State: Replica: 78369c5e3f6f4dbfa1fa5d1f511de286, State: Running, Role: LEADER
I20260811 07:01:33.853441 11089 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38847 } }
I20260811 07:01:33.854193 11071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:33.856604 11092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78369c5e3f6f4dbfa1fa5d1f511de286. Latest consensus state: current_term: 1 leader_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38847 } } }
I20260811 07:01:33.857095 11092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:33.858728 11103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:33.859020 11090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78369c5e3f6f4dbfa1fa5d1f511de286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38847 } } }
I20260811 07:01:33.859464 11090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:33.863358 11103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:33.872448 11103 catalog_manager.cc:1347] Generated new cluster ID: 1743345601874f9faf1bf3ca50f6a5be
I20260811 07:01:33.872560 11103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:33.889225 11103 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:33.889765 11103 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:33.902451 11103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78369c5e3f6f4dbfa1fa5d1f511de286: Generated new TSK 0
I20260811 07:01:33.903506 11103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:33.913520 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10955
W20260811 07:01:33.914711 11065 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.91:46625
W20260811 07:01:33.915448 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 10955: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:33.915611 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 10955
2026-08-11T07:01:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2277 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:34 dist-test-slave-7vjv krb5kdc[11123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:34 dist-test-slave-7vjv krb5kdc[11123](info): set up 2 sockets
Aug 11 07:01:34 dist-test-slave-7vjv krb5kdc[11123](info): commencing operation
krb5kdc: starting...
W20260811 07:01:35.325732 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.361s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:35 dist-test-slave-7vjv krb5kdc[11123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:35.661484 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36977
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:33979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:36977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:36.015389 11144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:36.015734 11144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:36.015925 11144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:36.023108 11144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:36.023270 11144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:36.023417 11144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:36.023527 11144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:36.023617 11144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:36.034061 11144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:33979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:36977
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:36977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:36.035844 11144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:36.038002 11144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:36.050882 11159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:36.050983 11157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:36.050945 11156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:36.051664 11144 server_base.cc:1034] running on GCE node
I20260811 07:01:36.052544 11144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:36.053963 11144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:36.055168 11144 hybrid_clock.cc:648] HybridClock initialized: now 1786431696055110 us; error 49 us; skew 500 ppm
I20260811 07:01:36.055550 11144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:36 dist-test-slave-7vjv krb5kdc[11123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431696, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:36.058676 11144 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:36.060050 11144 webserver.cc:460] Webserver started at http://127.4.40.62:41575/ using document root <none> and password file <none>
I20260811 07:01:36.060746 11144 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:36.060851 11144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:36.061103 11144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:36.063998 11144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "8645fdcb93494c3c97afcdd67742becd"
format_stamp: "Formatted at 2026-08-11 07:01:36 on dist-test-slave-7vjv"
server_key: "6d9c0179d1bf4d5f9260da048a69d410"
server_key_iv: "5fbb13709ffb7aff5cf7f247b4b8289d"
server_key_version: "encryptionkey@0"
I20260811 07:01:36.064761 11144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "8645fdcb93494c3c97afcdd67742becd"
format_stamp: "Formatted at 2026-08-11 07:01:36 on dist-test-slave-7vjv"
server_key: "6d9c0179d1bf4d5f9260da048a69d410"
server_key_iv: "5fbb13709ffb7aff5cf7f247b4b8289d"
server_key_version: "encryptionkey@0"
I20260811 07:01:36.070050 11144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 07:01:36.073654 11171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:36.074980 11144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 07:01:36.075191 11144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "8645fdcb93494c3c97afcdd67742becd"
format_stamp: "Formatted at 2026-08-11 07:01:36 on dist-test-slave-7vjv"
server_key: "6d9c0179d1bf4d5f9260da048a69d410"
server_key_iv: "5fbb13709ffb7aff5cf7f247b4b8289d"
server_key_version: "encryptionkey@0"
I20260811 07:01:36.075371 11144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:36.101552 11144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:36.105579 11144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:36.105870 11144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:36.124913 11144 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:36977
I20260811 07:01:36.124963 11254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:36977 every 8 connection(s)
I20260811 07:01:36.126276 11144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:36.129842 11255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:36.134042 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11144
I20260811 07:01:36.134243 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:36.134789 4256 external_mini_cluster.cc:1428] Setting key 47b62b53fb956775b84af02ea043fe3a
I20260811 07:01:36.140945 11255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd: Bootstrap starting.
I20260811 07:01:36.143970 11255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:36.144994 11255 log.cc:824] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:36.148447 11255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd: No bootstrap required, opened a new log
Aug 11 07:01:36 dist-test-slave-7vjv krb5kdc[11123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:36.159214 11255 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8645fdcb93494c3c97afcdd67742becd" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36977 } }
I20260811 07:01:36.159653 11255 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:36.159766 11255 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8645fdcb93494c3c97afcdd67742becd, State: Initialized, Role: FOLLOWER
I20260811 07:01:36.160025 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:36.136662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38423 (local address 127.4.40.62:36977)
0811 07:01:36.137468 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:36.137479 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:36.137530 (+ 51us) server_negotiation.cc:407] Connection header received
0811 07:01:36.139721 (+ 2191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:36.139742 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:36.140088 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:36.140519 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:36.142101 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:36.143406 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:36.144812 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:36.145678 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:36.145895 (+ 217us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:36.150971 (+ 5076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:36.150982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:36.151005 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:36.151062 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:36.155344 (+ 4282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:36.156224 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:36.156232 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:36.156240 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:36.156447 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:36.156904 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:36.156909 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:36.156913 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:36.157635 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:36.157960 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:36.158308 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 07:01:36.158628 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":391,"threads_started":1}
I20260811 07:01:36.160447 11255 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8645fdcb93494c3c97afcdd67742becd" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36977 } }
I20260811 07:01:36.160754 11255 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:36.160910 11255 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:36.161108 11255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:36.163851 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:36.163841 11255 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8645fdcb93494c3c97afcdd67742becd" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36977 } }
I20260811 07:01:36.164390 11255 leader_election.cc:302] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8645fdcb93494c3c97afcdd67742becd; no voters:
I20260811 07:01:36.165027 11255 leader_election.cc:288] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:36.165174 11266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:36.166446 11266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [term 1 LEADER]: Becoming Leader. State: Replica: 8645fdcb93494c3c97afcdd67742becd, State: Running, Role: LEADER
I20260811 07:01:36.167255 11255 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:36.167165 11266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8645fdcb93494c3c97afcdd67742becd" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36977 } }
I20260811 07:01:36.170642 11267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8645fdcb93494c3c97afcdd67742becd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8645fdcb93494c3c97afcdd67742becd" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36977 } } }
I20260811 07:01:36.171063 11267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:36.171628 11269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8645fdcb93494c3c97afcdd67742becd. Latest consensus state: current_term: 1 leader_uuid: "8645fdcb93494c3c97afcdd67742becd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8645fdcb93494c3c97afcdd67742becd" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 36977 } } }
I20260811 07:01:36.171964 11269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:36.172168 11285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:36.189899 11285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:36.212145 11285 catalog_manager.cc:1347] Generated new cluster ID: 7885cd7ff9c4419cba0b228443e4acbc
I20260811 07:01:36.212397 11285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:36.221613 11263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:36.193142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44031 (local address 127.4.40.62:36977)
0811 07:01:36.193341 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:36.193347 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:36.200653 (+ 7306us) server_negotiation.cc:407] Connection header received
0811 07:01:36.201250 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:36.201260 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:36.201371 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:36.201466 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:36.204450 (+ 2984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:36.206082 (+ 1632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:36.208527 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:36.209245 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:36.209389 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:36.213134 (+ 3745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:36.213141 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:36.213146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:36.213196 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:36.217195 (+ 3999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:36.218279 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:36.218285 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:36.218290 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:36.218446 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:36.219083 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:36.219088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:36.219091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:36.219489 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:36.219785 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:36.221034 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0811 07:01:36.221230 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:01:36.228569 11285 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:36.230360 11285 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:36.241360 11285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8645fdcb93494c3c97afcdd67742becd: Generated new TSK 0
I20260811 07:01:36.242255 11285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:36.249315 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11144
I20260811 07:01:36.250557 11248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:36.346352 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11144
W20260811 07:01:36.376871 11121 connection.cc:535] client connection to 127.4.40.62:36977 recv error: Network error: failed to read from TLS socket (remote: 127.4.40.62:36977): Connection reset by peer (error 104)
2026-08-11T07:01:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:36 dist-test-slave-7vjv krb5kdc[11321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:36 dist-test-slave-7vjv krb5kdc[11321](info): set up 2 sockets
Aug 11 07:01:36 dist-test-slave-7vjv krb5kdc[11321](info): commencing operation
krb5kdc: starting...
W20260811 07:01:37.753057 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.357s user 0.008s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:37 dist-test-slave-7vjv krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431697, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:38.083985 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45973
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:38.431530 11347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:38.431802 11347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:38.431977 11347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:38.439065 11347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:38.439170 11347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:38.439280 11347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:38.439394 11347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:38.439481 11347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:38.448999 11347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45973
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:38.450276 11347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:38.451668 11347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:38.463680 11355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:38.463645 11360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:38.463732 11357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:38.465011 11347 server_base.cc:1034] running on GCE node
I20260811 07:01:38.465698 11347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:38.467033 11347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:38.468247 11347 hybrid_clock.cc:648] HybridClock initialized: now 1786431698468222 us; error 42 us; skew 500 ppm
I20260811 07:01:38.468585 11347 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:38 dist-test-slave-7vjv krb5kdc[11321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431698, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:38.476387 11347 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:38.477939 11347 webserver.cc:460] Webserver started at http://127.4.40.62:41295/ using document root <none> and password file <none>
I20260811 07:01:38.478632 11347 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:38.478783 11347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:38.479096 11347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:38.482111 11347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "64ccc6523a7649bfbcb666cd317ceb2f"
format_stamp: "Formatted at 2026-08-11 07:01:38 on dist-test-slave-7vjv"
server_key: "9d724b7d2da71ccac945b594360dfe90"
server_key_iv: "0b10ef8417fbc11b5c56995878e67113"
server_key_version: "encryptionkey@0"
I20260811 07:01:38.482867 11347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "64ccc6523a7649bfbcb666cd317ceb2f"
format_stamp: "Formatted at 2026-08-11 07:01:38 on dist-test-slave-7vjv"
server_key: "9d724b7d2da71ccac945b594360dfe90"
server_key_iv: "0b10ef8417fbc11b5c56995878e67113"
server_key_version: "encryptionkey@0"
I20260811 07:01:38.487973 11347 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 07:01:38.491302 11376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:38.492624 11347 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:38.492807 11347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "64ccc6523a7649bfbcb666cd317ceb2f"
format_stamp: "Formatted at 2026-08-11 07:01:38 on dist-test-slave-7vjv"
server_key: "9d724b7d2da71ccac945b594360dfe90"
server_key_iv: "0b10ef8417fbc11b5c56995878e67113"
server_key_version: "encryptionkey@0"
I20260811 07:01:38.493023 11347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:38.505343 11347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:38.509475 11347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:38.509764 11347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:38.527608 11347 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45973
I20260811 07:01:38.527653 11480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45973 every 8 connection(s)
I20260811 07:01:38.529019 11347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:38.532770 11481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:38.538425 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11347
I20260811 07:01:38.538625 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:38.539103 4256 external_mini_cluster.cc:1428] Setting key b7586157078d36e0e36f9fbe1c27d4ba
I20260811 07:01:38.544349 11481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f: Bootstrap starting.
I20260811 07:01:38.547995 11481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:38.549162 11481 log.cc:824] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:38.552352 11481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f: No bootstrap required, opened a new log
I20260811 07:01:38.553697 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:38.540936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38547 (local address 127.4.40.62:45973)
0811 07:01:38.541783 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:38.541797 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:38.541852 (+ 55us) server_negotiation.cc:407] Connection header received
0811 07:01:38.544139 (+ 2287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:38.544165 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:38.544633 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:38.545033 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:38.545504 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:38.547394 (+ 1890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:38.548720 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:38.549848 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:38.549984 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:38.551358 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:38.551371 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:38.551389 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:38.552079 (+ 690us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:38.552182 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:38.552218 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:38.552379 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:38.552673 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 07:01:38.552832 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":589,"thread_start_us":339,"threads_started":1}
I20260811 07:01:38.557648 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:38.560912 11481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45973 } }
I20260811 07:01:38.561267 11481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:38.561371 11481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64ccc6523a7649bfbcb666cd317ceb2f, State: Initialized, Role: FOLLOWER
I20260811 07:01:38.562049 11481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45973 } }
I20260811 07:01:38.562335 11481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:38.562484 11481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:38.562677 11481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:38.565281 11481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45973 } }
I20260811 07:01:38.565840 11481 leader_election.cc:302] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64ccc6523a7649bfbcb666cd317ceb2f; no voters:
I20260811 07:01:38.566535 11481 leader_election.cc:288] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:38.566653 11501 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:38.568647 11501 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [term 1 LEADER]: Becoming Leader. State: Replica: 64ccc6523a7649bfbcb666cd317ceb2f, State: Running, Role: LEADER
I20260811 07:01:38.569154 11481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:38.569289 11501 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45973 } }
I20260811 07:01:38.571269 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:38.562025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57775 (local address 127.4.40.62:45973)
0811 07:01:38.562249 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:38.562258 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:38.564682 (+ 2424us) server_negotiation.cc:407] Connection header received
0811 07:01:38.564827 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:38.564831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:38.564883 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:38.564955 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:38.565649 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:38.566438 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:38.568394 (+ 1956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:38.569088 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:38.569245 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:38.570494 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:38.570498 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:38.570501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:38.570518 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:38.570573 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:38.570588 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:38.570707 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:38.570901 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 07:01:38.570950 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:01:38.577812 11504 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45973 } } }
I20260811 07:01:38.577783 11505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ccc6523a7649bfbcb666cd317ceb2f. Latest consensus state: current_term: 1 leader_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ccc6523a7649bfbcb666cd317ceb2f" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45973 } } }
I20260811 07:01:38.578292 11504 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:38.578356 11505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:38.579283 11514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:38.586393 11514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:38.595283 11514 catalog_manager.cc:1347] Generated new cluster ID: f65c19c911cd43acbb0a06fa1be7ba4b
I20260811 07:01:38.595402 11514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:38.607601 11514 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:38.608909 11514 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:38.628162 11514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64ccc6523a7649bfbcb666cd317ceb2f: Generated new TSK 0
I20260811 07:01:38.628978 11514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:38.634374 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11347
W20260811 07:01:38.635543 11472 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38547
W20260811 07:01:38.636338 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11347: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:38.636488 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11347
W20260811 07:01:38.663491 11319 connection.cc:535] client connection to 127.4.40.62:45973 recv error: Network error: failed to read from TLS socket (remote: 127.4.40.62:45973): Connection reset by peer (error 104)
2026-08-11T07:01:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2286 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:38 dist-test-slave-7vjv krb5kdc[11534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:38 dist-test-slave-7vjv krb5kdc[11534](info): set up 2 sockets
Aug 11 07:01:38 dist-test-slave-7vjv krb5kdc[11534](info): commencing operation
krb5kdc: starting...
W20260811 07:01:40.014343 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.332s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:40 dist-test-slave-7vjv krb5kdc[11534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431700, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:40.355211 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45163
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:45163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:40.708549 11553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:40.708877 11553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:40.709071 11553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:40.716457 11553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:40.716598 11553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:40.716711 11553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:40.716849 11553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:40.716948 11553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:40.726680 11553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:45163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:45163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:40.728035 11553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:40.729453 11553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:40.741874 11558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:40.741938 11560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:40.741838 11563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:40.742690 11553 server_base.cc:1034] running on GCE node
I20260811 07:01:40.743551 11553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:40.745069 11553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:40.746320 11553 hybrid_clock.cc:648] HybridClock initialized: now 1786431700746245 us; error 52 us; skew 500 ppm
I20260811 07:01:40.746692 11553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:40 dist-test-slave-7vjv krb5kdc[11534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431700, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:40.750253 11553 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:40.751863 11553 webserver.cc:460] Webserver started at http://127.4.40.62:39771/ using document root <none> and password file <none>
I20260811 07:01:40.752609 11553 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:40.752730 11553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:40.753042 11553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:40.756224 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "35ce22550d184ce6962cdd6492b3631c"
format_stamp: "Formatted at 2026-08-11 07:01:40 on dist-test-slave-7vjv"
server_key: "bcd18097a2ec1d49a7ec805a0494d68e"
server_key_iv: "e00d42c7400c98156b5c2eda91af7463"
server_key_version: "encryptionkey@0"
I20260811 07:01:40.757047 11553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "35ce22550d184ce6962cdd6492b3631c"
format_stamp: "Formatted at 2026-08-11 07:01:40 on dist-test-slave-7vjv"
server_key: "bcd18097a2ec1d49a7ec805a0494d68e"
server_key_iv: "e00d42c7400c98156b5c2eda91af7463"
server_key_version: "encryptionkey@0"
I20260811 07:01:40.762635 11553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 07:01:40.766211 11577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:40.767525 11553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:01:40.767755 11553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "35ce22550d184ce6962cdd6492b3631c"
format_stamp: "Formatted at 2026-08-11 07:01:40 on dist-test-slave-7vjv"
server_key: "bcd18097a2ec1d49a7ec805a0494d68e"
server_key_iv: "e00d42c7400c98156b5c2eda91af7463"
server_key_version: "encryptionkey@0"
I20260811 07:01:40.767971 11553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:40.774178 11553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:40.778304 11553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:40.778709 11553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:40.799084 11553 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:45163
I20260811 07:01:40.799134 11670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:45163 every 8 connection(s)
I20260811 07:01:40.800733 11553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:40.804473 11676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:40.810132 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11553
I20260811 07:01:40.810313 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:40.810787 4256 external_mini_cluster.cc:1428] Setting key 96fbaabd88c637638dc6aa702ebefca4
I20260811 07:01:40.816046 11676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c: Bootstrap starting.
I20260811 07:01:40.819782 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:40.812669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52067 (local address 127.4.40.62:45163)
0811 07:01:40.813602 (+ 933us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:40.813617 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:40.813670 (+ 53us) server_negotiation.cc:407] Connection header received
0811 07:01:40.816019 (+ 2349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:40.816053 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:40.816481 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:40.816908 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:40.817881 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:40.817891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:40.817908 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:40.817928 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:40.818013 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:40.818042 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:40.818275 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:40.818416 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 07:01:40.818598 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":413,"threads_started":1}
I20260811 07:01:40.820122 11676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:40.821426 11676 log.cc:824] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:40.823707 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:40.824657 11676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c: No bootstrap required, opened a new log
I20260811 07:01:40.831600 11679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:40.828892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49881 (local address 127.4.40.62:45163)
0811 07:01:40.829151 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:40.829159 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:40.829731 (+ 572us) server_negotiation.cc:407] Connection header received
0811 07:01:40.829846 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:40.829851 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:40.829913 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:40.830065 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:40.830955 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:40.830961 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:40.830983 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:40.831001 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:40.831051 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:40.831069 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:40.831198 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:40.831243 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 07:01:40.831294 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 07:01:40.834450 11676 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ce22550d184ce6962cdd6492b3631c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45163 } }
I20260811 07:01:40.834791 11676 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:40.834863 11676 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35ce22550d184ce6962cdd6492b3631c, State: Initialized, Role: FOLLOWER
I20260811 07:01:40.835333 11676 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ce22550d184ce6962cdd6492b3631c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45163 } }
I20260811 07:01:40.835625 11676 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:40.835788 11676 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:40.835991 11676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:40.838295 11676 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ce22550d184ce6962cdd6492b3631c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45163 } }
I20260811 07:01:40.838785 11676 leader_election.cc:302] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35ce22550d184ce6962cdd6492b3631c; no voters:
I20260811 07:01:40.839370 11676 leader_election.cc:288] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:40.839468 11690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:40.840324 11690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [term 1 LEADER]: Becoming Leader. State: Replica: 35ce22550d184ce6962cdd6492b3631c, State: Running, Role: LEADER
I20260811 07:01:40.840978 11690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ce22550d184ce6962cdd6492b3631c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45163 } }
I20260811 07:01:40.841837 11676 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:40.844337 11694 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35ce22550d184ce6962cdd6492b3631c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ce22550d184ce6962cdd6492b3631c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45163 } } }
I20260811 07:01:40.844761 11694 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:40.846354 11707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:40.846309 11696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35ce22550d184ce6962cdd6492b3631c. Latest consensus state: current_term: 1 leader_uuid: "35ce22550d184ce6962cdd6492b3631c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35ce22550d184ce6962cdd6492b3631c" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 45163 } } }
I20260811 07:01:40.846624 11696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:40.851456 11707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:40.860874 11707 catalog_manager.cc:1347] Generated new cluster ID: 306c9244e16344ae851a1bec1986a936
I20260811 07:01:40.860982 11707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:40.878161 11707 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:40.878643 11707 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:40.889215 11707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35ce22550d184ce6962cdd6492b3631c: Generated new TSK 0
I20260811 07:01:40.889977 11707 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:40.897416 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11553
W20260811 07:01:40.898535 11666 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52067
W20260811 07:01:40.899257 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11553: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:40.899447 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11553
2026-08-11T07:01:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2266 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:41 dist-test-slave-7vjv krb5kdc[11729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:41 dist-test-slave-7vjv krb5kdc[11729](info): set up 2 sockets
Aug 11 07:01:41 dist-test-slave-7vjv krb5kdc[11729](info): commencing operation
krb5kdc: starting...
W20260811 07:01:42.287112 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.338s user 0.004s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:42 dist-test-slave-7vjv krb5kdc[11729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:42.639670 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42387
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:45533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:42387
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:42.980139 11756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:42.980479 11756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:42.980664 11756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:42.987820 11756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:42.987953 11756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:42.988061 11756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:42.988201 11756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:42.988269 11756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:42.998005 11756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:45533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:42387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:42387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:42.999300 11756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:43.000674 11756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:43.012925 11763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:43.012858 11767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:43.012893 11764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:43.013994 11756 server_base.cc:1034] running on GCE node
I20260811 07:01:43.014740 11756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:43.016052 11756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:43.017314 11756 hybrid_clock.cc:648] HybridClock initialized: now 1786431703017273 us; error 34 us; skew 500 ppm
I20260811 07:01:43.017668 11756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:43 dist-test-slave-7vjv krb5kdc[11729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431703, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:43.020812 11756 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:43.022334 11756 webserver.cc:460] Webserver started at http://127.4.40.62:46495/ using document root <none> and password file <none>
I20260811 07:01:43.023013 11756 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:43.023162 11756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:43.023468 11756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:43.026537 11756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "a77104bb2759481b8d1813f26a2215df"
format_stamp: "Formatted at 2026-08-11 07:01:43 on dist-test-slave-7vjv"
server_key: "bcaeb9f3b632362cc339119c3655e4f8"
server_key_iv: "adcb8eee36f66f918f7a9567effe70a1"
server_key_version: "encryptionkey@0"
I20260811 07:01:43.027307 11756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "a77104bb2759481b8d1813f26a2215df"
format_stamp: "Formatted at 2026-08-11 07:01:43 on dist-test-slave-7vjv"
server_key: "bcaeb9f3b632362cc339119c3655e4f8"
server_key_iv: "adcb8eee36f66f918f7a9567effe70a1"
server_key_version: "encryptionkey@0"
I20260811 07:01:43.032523 11756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 07:01:43.035733 11775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:43.037071 11756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:43.037288 11756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "a77104bb2759481b8d1813f26a2215df"
format_stamp: "Formatted at 2026-08-11 07:01:43 on dist-test-slave-7vjv"
server_key: "bcaeb9f3b632362cc339119c3655e4f8"
server_key_iv: "adcb8eee36f66f918f7a9567effe70a1"
server_key_version: "encryptionkey@0"
I20260811 07:01:43.037483 11756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:43.052372 11756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:43.056663 11756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:43.057026 11756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:43.075690 11756 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:42387
I20260811 07:01:43.075728 11860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:42387 every 8 connection(s)
I20260811 07:01:43.077301 11756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:43.081421 11863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:43.084725 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11756
I20260811 07:01:43.084913 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:43.085479 4256 external_mini_cluster.cc:1428] Setting key 968493d99c181c06e9133bb61c7fced2
I20260811 07:01:43.094112 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df: Bootstrap starting.
I20260811 07:01:43.098583 11863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:43.100180 11863 log.cc:824] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df: Log is configured to *not* fsync() on all Append() calls
Aug 11 07:01:43 dist-test-slave-7vjv krb5kdc[11729](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431702, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:43.103583 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df: No bootstrap required, opened a new log
I20260811 07:01:43.110809 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:43.087398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34879 (local address 127.4.40.62:42387)
0811 07:01:43.088236 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:43.088248 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:43.088290 (+ 42us) server_negotiation.cc:407] Connection header received
0811 07:01:43.089964 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:43.089986 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:43.090386 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:43.090722 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:43.092590 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:43.093847 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:43.095693 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:43.096445 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:43.096606 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:43.102071 (+ 5465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:43.102082 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:43.102111 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:43.102158 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:43.106299 (+ 4141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:43.107043 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:43.107052 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:43.107065 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:43.107288 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:43.107625 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:43.107630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:43.107634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:43.108395 (+ 761us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:43.108690 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:43.109013 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 07:01:43.109451 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":371,"threads_started":1}
I20260811 07:01:43.114419 11863 raft_consensus.cc:348] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77104bb2759481b8d1813f26a2215df" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42387 } }
I20260811 07:01:43.114851 11863 raft_consensus.cc:374] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:43.114986 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:43.114979 11863 raft_consensus.cc:729] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a77104bb2759481b8d1813f26a2215df, State: Initialized, Role: FOLLOWER
I20260811 07:01:43.115588 11863 consensus_queue.cc:260] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77104bb2759481b8d1813f26a2215df" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42387 } }
I20260811 07:01:43.115826 11863 raft_consensus.cc:388] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:43.116005 11863 raft_consensus.cc:482] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:43.116202 11863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:43.118335 11863 raft_consensus.cc:504] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77104bb2759481b8d1813f26a2215df" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42387 } }
I20260811 07:01:43.118810 11863 leader_election.cc:302] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a77104bb2759481b8d1813f26a2215df; no voters:
I20260811 07:01:43.119481 11863 leader_election.cc:288] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:43.119722 11874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:43.122129 11863 sys_catalog.cc:564] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:43.125092 11874 raft_consensus.cc:686] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [term 1 LEADER]: Becoming Leader. State: Replica: a77104bb2759481b8d1813f26a2215df, State: Running, Role: LEADER
I20260811 07:01:43.126036 11874 consensus_queue.cc:237] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77104bb2759481b8d1813f26a2215df" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42387 } }
I20260811 07:01:43.129576 11875 sys_catalog.cc:455] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a77104bb2759481b8d1813f26a2215df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77104bb2759481b8d1813f26a2215df" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42387 } } }
I20260811 07:01:43.130035 11875 sys_catalog.cc:458] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:43.130672 11882 sys_catalog.cc:455] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [sys.catalog]: SysCatalogTable state changed. Reason: New leader a77104bb2759481b8d1813f26a2215df. Latest consensus state: current_term: 1 leader_uuid: "a77104bb2759481b8d1813f26a2215df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a77104bb2759481b8d1813f26a2215df" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 42387 } } }
I20260811 07:01:43.130991 11882 sys_catalog.cc:458] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:43.139174 11899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:43.144620 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:43.122178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50903 (local address 127.4.40.62:42387)
0811 07:01:43.122398 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:43.122407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:43.125154 (+ 2747us) server_negotiation.cc:407] Connection header received
0811 07:01:43.125430 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:43.125437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:43.125555 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:43.125642 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:43.127865 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:43.129079 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:43.131084 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:43.132048 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:43.132219 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:43.136182 (+ 3963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:43.136206 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:43.136213 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:43.136261 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:43.140904 (+ 4643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:43.142155 (+ 1251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:43.142163 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:43.142167 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:43.142325 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:43.142838 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:43.142842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:43.142845 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:43.143278 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:43.143523 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:43.143952 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 07:01:43.144249 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 07:01:43.145581 11899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:43.155061 11899 catalog_manager.cc:1347] Generated new cluster ID: 797462f492134e8589400c40f8636cab
I20260811 07:01:43.155208 11899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:43.169941 11899 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:43.171350 11899 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:43.179870 11899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a77104bb2759481b8d1813f26a2215df: Generated new TSK 0
I20260811 07:01:43.180871 11899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:43.193148 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11756
I20260811 07:01:43.194334 11852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:43.284623 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11756
2026-08-11T07:01:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2378 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:43 dist-test-slave-7vjv krb5kdc[11913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:43 dist-test-slave-7vjv krb5kdc[11913](info): set up 2 sockets
Aug 11 07:01:43 dist-test-slave-7vjv krb5kdc[11913](info): commencing operation
krb5kdc: starting...
W20260811 07:01:44.698523 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.371s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:44 dist-test-slave-7vjv krb5kdc[11913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:45.050823 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:34869
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:34869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:45.401855 11939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:45.402179 11939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:45.402375 11939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:45.409780 11939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:45.409957 11939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:45.410058 11939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:45.410159 11939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:45.410256 11939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:45.420377 11939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:34869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:34869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:45.421650 11939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:45.423051 11939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:45.435417 11948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:45.435494 11952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:45.435480 11949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:45.437140 11939 server_base.cc:1034] running on GCE node
I20260811 07:01:45.437822 11939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:45.439163 11939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:45.440359 11939 hybrid_clock.cc:648] HybridClock initialized: now 1786431705440305 us; error 53 us; skew 500 ppm
I20260811 07:01:45.440706 11939 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:45 dist-test-slave-7vjv krb5kdc[11913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431705, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:45.443805 11939 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:45.445307 11939 webserver.cc:460] Webserver started at http://127.4.40.62:41725/ using document root <none> and password file <none>
I20260811 07:01:45.445977 11939 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:45.446110 11939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:45.446415 11939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:45.449489 11939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "566bcf14f65b4fe39841820437594ecb"
format_stamp: "Formatted at 2026-08-11 07:01:45 on dist-test-slave-7vjv"
server_key: "626bc55bb95ed49b8b93cae80d674706"
server_key_iv: "58031e576b0e7138a1d42237dfaa4381"
server_key_version: "encryptionkey@0"
I20260811 07:01:45.450513 11939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "566bcf14f65b4fe39841820437594ecb"
format_stamp: "Formatted at 2026-08-11 07:01:45 on dist-test-slave-7vjv"
server_key: "626bc55bb95ed49b8b93cae80d674706"
server_key_iv: "58031e576b0e7138a1d42237dfaa4381"
server_key_version: "encryptionkey@0"
I20260811 07:01:45.455765 11939 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 07:01:45.459085 11964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:45.460428 11939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 07:01:45.460606 11939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "566bcf14f65b4fe39841820437594ecb"
format_stamp: "Formatted at 2026-08-11 07:01:45 on dist-test-slave-7vjv"
server_key: "626bc55bb95ed49b8b93cae80d674706"
server_key_iv: "58031e576b0e7138a1d42237dfaa4381"
server_key_version: "encryptionkey@0"
I20260811 07:01:45.460804 11939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:45.474344 11939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:45.478032 11939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:45.478363 11939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:45.497265 11939 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:34869
I20260811 07:01:45.497305 12048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:34869 every 8 connection(s)
I20260811 07:01:45.498720 11939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:45.502347 12050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:45.504361 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 11939
I20260811 07:01:45.504531 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:45.505008 4256 external_mini_cluster.cc:1428] Setting key 4841ef719374feb1a1b9e0c2274d6d2c
I20260811 07:01:45.514716 12050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb: Bootstrap starting.
I20260811 07:01:45.518481 12050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:45.519274 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:45.506913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46651 (local address 127.4.40.62:34869)
0811 07:01:45.507799 (+ 886us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:45.507810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:45.507854 (+ 44us) server_negotiation.cc:407] Connection header received
0811 07:01:45.509618 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:45.509639 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:45.510038 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:45.510356 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:45.511115 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:45.512594 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:45.514673 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:45.515452 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:45.515594 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:45.516915 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:45.516922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:45.516934 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:45.517471 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:45.517551 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:45.517575 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:45.517668 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:45.517908 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 07:01:45.518190 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":419,"threads_started":1}
I20260811 07:01:45.519734 12050 log.cc:824] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:45.522930 12050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb: No bootstrap required, opened a new log
I20260811 07:01:45.523461 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:45.533810 12050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566bcf14f65b4fe39841820437594ecb" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 34869 } }
I20260811 07:01:45.534354 12050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:45.534476 12050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 566bcf14f65b4fe39841820437594ecb, State: Initialized, Role: FOLLOWER
I20260811 07:01:45.535198 12050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566bcf14f65b4fe39841820437594ecb" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 34869 } }
I20260811 07:01:45.535524 12050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:45.535732 12050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:45.535773 12055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:45.528918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45393 (local address 127.4.40.62:34869)
0811 07:01:45.529075 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:45.529079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:45.529455 (+ 376us) server_negotiation.cc:407] Connection header received
0811 07:01:45.529677 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:45.529681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:45.529758 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:45.529859 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:45.530540 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:45.531260 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:45.533154 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:45.533587 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:45.533676 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:45.534944 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:45.534952 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:45.534957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:45.534990 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:45.535036 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:45.535053 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:45.535124 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:45.535307 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 07:01:45.535375 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 07:01:45.535944 12050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:45.538682 12050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566bcf14f65b4fe39841820437594ecb" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 34869 } }
I20260811 07:01:45.539276 12050 leader_election.cc:302] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 566bcf14f65b4fe39841820437594ecb; no voters:
I20260811 07:01:45.539911 12050 leader_election.cc:288] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:45.540050 12066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:45.540843 12066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [term 1 LEADER]: Becoming Leader. State: Replica: 566bcf14f65b4fe39841820437594ecb, State: Running, Role: LEADER
I20260811 07:01:45.541488 12066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566bcf14f65b4fe39841820437594ecb" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 34869 } }
I20260811 07:01:45.542296 12050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:45.544348 12070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "566bcf14f65b4fe39841820437594ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566bcf14f65b4fe39841820437594ecb" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 34869 } } }
I20260811 07:01:45.544775 12070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:45.544500 12072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 566bcf14f65b4fe39841820437594ecb. Latest consensus state: current_term: 1 leader_uuid: "566bcf14f65b4fe39841820437594ecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "566bcf14f65b4fe39841820437594ecb" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 34869 } } }
I20260811 07:01:45.544989 12072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:45.547196 12079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:45.553560 12079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:45.562139 12079 catalog_manager.cc:1347] Generated new cluster ID: a21a783787ef484f9a4615756a043a34
I20260811 07:01:45.562264 12079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:45.570394 12079 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:45.571583 12079 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:45.579553 12079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 566bcf14f65b4fe39841820437594ecb: Generated new TSK 0
I20260811 07:01:45.580358 12079 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:45.590633 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11939
W20260811 07:01:45.591765 12041 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46651
W20260811 07:01:45.592572 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 11939: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:45.592729 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 11939
2026-08-11T07:01:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:45 dist-test-slave-7vjv krb5kdc[12111](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:45 dist-test-slave-7vjv krb5kdc[12111](info): set up 2 sockets
Aug 11 07:01:45 dist-test-slave-7vjv krb5kdc[12111](info): commencing operation
krb5kdc: starting...
W20260811 07:01:46.993557 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.351s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:47 dist-test-slave-7vjv krb5kdc[12111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:47.339279 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37377
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:44335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:37377
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:47.680866 12131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:47.681146 12131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:47.681249 12131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:47.688081 12131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:47.688194 12131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:47.688297 12131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:47.688370 12131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:47.688414 12131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:47.697530 12131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:44335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:37377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:37377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:47.698689 12131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:47.699973 12131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:47.711570 12150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:47.711647 12141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:47.711647 12143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:47.712621 12131 server_base.cc:1034] running on GCE node
I20260811 07:01:47.713399 12131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:47.714747 12131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:47.715948 12131 hybrid_clock.cc:648] HybridClock initialized: now 1786431707715898 us; error 38 us; skew 500 ppm
I20260811 07:01:47.716334 12131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:47 dist-test-slave-7vjv krb5kdc[12111](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431707, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:47.719297 12131 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:47.720841 12131 webserver.cc:460] Webserver started at http://127.4.40.62:34381/ using document root <none> and password file <none>
I20260811 07:01:47.721575 12131 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:47.721719 12131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:47.722040 12131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:47.725015 12131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "caa29e93f9e6489fae54b243737d903d"
format_stamp: "Formatted at 2026-08-11 07:01:47 on dist-test-slave-7vjv"
server_key: "c0256523804e537b45df9344703506b8"
server_key_iv: "1892195d4a7369d5cf6b33c395e148bd"
server_key_version: "encryptionkey@0"
I20260811 07:01:47.725759 12131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "caa29e93f9e6489fae54b243737d903d"
format_stamp: "Formatted at 2026-08-11 07:01:47 on dist-test-slave-7vjv"
server_key: "c0256523804e537b45df9344703506b8"
server_key_iv: "1892195d4a7369d5cf6b33c395e148bd"
server_key_version: "encryptionkey@0"
I20260811 07:01:47.730785 12131 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 07:01:47.734262 12161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:47.735486 12131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:01:47.735674 12131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "caa29e93f9e6489fae54b243737d903d"
format_stamp: "Formatted at 2026-08-11 07:01:47 on dist-test-slave-7vjv"
server_key: "c0256523804e537b45df9344703506b8"
server_key_iv: "1892195d4a7369d5cf6b33c395e148bd"
server_key_version: "encryptionkey@0"
I20260811 07:01:47.735857 12131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:47.741662 12131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:47.745611 12131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:47.745908 12131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:47.764290 12131 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:37377
I20260811 07:01:47.764309 12254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:37377 every 8 connection(s)
I20260811 07:01:47.765692 12131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:47.769266 12257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:47.772210 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12131
I20260811 07:01:47.772400 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:47.772948 4256 external_mini_cluster.cc:1428] Setting key ea0f4f09aa6479516ff5b96e5a1f2c92
I20260811 07:01:47.780552 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:47.774810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45813 (local address 127.4.40.62:37377)
0811 07:01:47.775589 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:47.775599 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:47.775639 (+ 40us) server_negotiation.cc:407] Connection header received
0811 07:01:47.777265 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:47.777285 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:47.777578 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:47.777908 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:47.778952 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:47.778957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:47.778968 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:47.778984 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:47.779049 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:47.779070 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:47.779234 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:47.779354 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 07:01:47.779624 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":372,"threads_started":1}
I20260811 07:01:47.781419 12257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d: Bootstrap starting.
I20260811 07:01:47.784231 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:47.785094 12257 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:47.786312 12257 log.cc:824] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:47.789522 12257 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d: No bootstrap required, opened a new log
I20260811 07:01:47.791699 12261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:47.789324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47619 (local address 127.4.40.62:37377)
0811 07:01:47.789497 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:47.789503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:47.789914 (+ 411us) server_negotiation.cc:407] Connection header received
0811 07:01:47.790039 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:47.790043 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:47.790097 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:47.790203 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:47.791076 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:47.791081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:47.791085 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 07:01:47.791103 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:47.791158 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 07:01:47.791177 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:47.791300 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:47.791348 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 07:01:47.791409 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 07:01:47.797689 12257 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa29e93f9e6489fae54b243737d903d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37377 } }
I20260811 07:01:47.798024 12257 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 07:01:47.798095 12257 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa29e93f9e6489fae54b243737d903d, State: Initialized, Role: FOLLOWER
I20260811 07:01:47.798667 12257 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa29e93f9e6489fae54b243737d903d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37377 } }
I20260811 07:01:47.798960 12257 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 07:01:47.799155 12257 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 07:01:47.799325 12257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:47.801561 12257 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa29e93f9e6489fae54b243737d903d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37377 } }
I20260811 07:01:47.802033 12257 leader_election.cc:302] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa29e93f9e6489fae54b243737d903d; no voters:
I20260811 07:01:47.802654 12257 leader_election.cc:288] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 07:01:47.802794 12273 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:47.803771 12273 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [term 1 LEADER]: Becoming Leader. State: Replica: caa29e93f9e6489fae54b243737d903d, State: Running, Role: LEADER
I20260811 07:01:47.804316 12273 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa29e93f9e6489fae54b243737d903d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37377 } }
I20260811 07:01:47.804728 12257 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:47.806787 12277 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa29e93f9e6489fae54b243737d903d. Latest consensus state: current_term: 1 leader_uuid: "caa29e93f9e6489fae54b243737d903d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa29e93f9e6489fae54b243737d903d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37377 } } }
I20260811 07:01:47.807152 12277 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:47.807652 12276 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa29e93f9e6489fae54b243737d903d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa29e93f9e6489fae54b243737d903d" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 37377 } } }
I20260811 07:01:47.808009 12276 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:47.812829 12287 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 07:01:47.818567 12287 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:47.827527 12287 catalog_manager.cc:1347] Generated new cluster ID: 27d5923aad3a4d9ca1543c1dfb16ad90
I20260811 07:01:47.827672 12287 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:47.849742 12287 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:47.850178 12287 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:47.862180 12287 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa29e93f9e6489fae54b243737d903d: Generated new TSK 0
I20260811 07:01:47.862965 12287 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:47.879015 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12131
W20260811 07:01:47.880146 12246 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:45813
W20260811 07:01:47.880810 4256 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12131: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 07:01:47.880978 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12131
2026-08-11T07:01:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2289 ms)
[----------] 18 tests from AuthTokenIssuingTest (41993 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:48 dist-test-slave-7vjv krb5kdc[12316](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:48 dist-test-slave-7vjv krb5kdc[12316](info): set up 2 sockets
Aug 11 07:01:48 dist-test-slave-7vjv krb5kdc[12316](info): commencing operation
krb5kdc: starting...
W20260811 07:01:49.278059 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.346s user 0.000s sys 0.053s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:49 dist-test-slave-7vjv krb5kdc[12316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:49.628369 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:35089
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:35089,127.4.40.61:36227,127.4.40.60:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:49.971808 12344 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:01:49.972186 12344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:49.972301 12344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:49.972456 12344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:49.979746 12344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:49.979830 12344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:49.979944 12344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:49.980036 12344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:49.980165 12344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:49.989863 12344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:35089,127.4.40.61:36227,127.4.40.60:44823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:35089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:49.991166 12344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:49.992585 12344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:50.004796 12354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:50.004858 12357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:50.004882 12359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:50.005767 12344 server_base.cc:1034] running on GCE node
I20260811 07:01:50.006568 12344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:50.007925 12344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:50.009142 12344 hybrid_clock.cc:648] HybridClock initialized: now 1786431710009073 us; error 45 us; skew 500 ppm
I20260811 07:01:50.009477 12344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:50 dist-test-slave-7vjv krb5kdc[12316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:50.016445 12344 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:50.017861 12344 webserver.cc:460] Webserver started at http://127.4.40.62:42085/ using document root <none> and password file <none>
I20260811 07:01:50.018610 12344 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:50.018754 12344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:50.019074 12344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:50.022109 12344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "7f91b479b6b047c3b894cebbaa02bc15"
format_stamp: "Formatted at 2026-08-11 07:01:50 on dist-test-slave-7vjv"
server_key: "ec3ec51efd80d4666476a4f492bb1608"
server_key_iv: "89973d33a9f7a150dad80949aa0efcb6"
server_key_version: "encryptionkey@0"
I20260811 07:01:50.023028 12344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "7f91b479b6b047c3b894cebbaa02bc15"
format_stamp: "Formatted at 2026-08-11 07:01:50 on dist-test-slave-7vjv"
server_key: "ec3ec51efd80d4666476a4f492bb1608"
server_key_iv: "89973d33a9f7a150dad80949aa0efcb6"
server_key_version: "encryptionkey@0"
I20260811 07:01:50.028029 12344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 07:01:50.031335 12370 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:50.032634 12344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:50.032801 12344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "7f91b479b6b047c3b894cebbaa02bc15"
format_stamp: "Formatted at 2026-08-11 07:01:50 on dist-test-slave-7vjv"
server_key: "ec3ec51efd80d4666476a4f492bb1608"
server_key_iv: "89973d33a9f7a150dad80949aa0efcb6"
server_key_version: "encryptionkey@0"
I20260811 07:01:50.033002 12344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:50.054188 12344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:50.058490 12344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:50.058759 12344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:50.077680 12344 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:35089
I20260811 07:01:50.077718 12468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:35089 every 8 connection(s)
I20260811 07:01:50.079025 12344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:50.082928 12469 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:50.083642 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12344
I20260811 07:01:50.083868 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:50.084753 4256 external_mini_cluster.cc:1428] Setting key c614ef34d7aafe4c4e5c8edeb8913c22
I20260811 07:01:50.092864 12469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:50 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:50.116343 12474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.096637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43644 (local address 127.4.40.62:35089)
0811 07:01:50.098283 (+ 1646us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:50.098295 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:50.098351 (+ 56us) server_negotiation.cc:407] Connection header received
0811 07:01:50.099866 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:50.099872 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:50.100073 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:50.100341 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:50.101659 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.102989 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.104453 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.105492 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.105709 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.109278 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:50.109285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:50.109295 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:50.109328 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:50.112075 (+ 2747us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.113037 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.113042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.113045 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.113202 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.113664 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.113667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.113669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.114124 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:50.114280 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:50.115006 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 07:01:50.115263 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":1430,"thread_start_us":1230,"threads_started":1}
I20260811 07:01:50.116433 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.096636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35089 (local address 127.0.0.1:43644)
0811 07:01:50.097536 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.097582 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:50.099469 (+ 1887us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:50.100355 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:50.100361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:50.101081 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:50.101423 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.101438 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.103154 (+ 1716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.103158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.104309 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.104321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.105713 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.105717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.105932 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.106627 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:50.106653 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:50.109060 (+ 2407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:50.112244 (+ 3184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.112259 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.112271 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.112909 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.113330 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.113336 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.113343 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.113555 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.114284 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:50.114296 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:50.114620 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 07:01:50.114882 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 07:01:50.115457 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":645,"mutex_wait_us":128,"thread_start_us":440,"threads_started":1}
I20260811 07:01:50.119271 12469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:50.120426 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.120042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50762)
0811 07:01:50.120234 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.120324 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260811 07:01:50.120702 12372 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.40.61@KRBTEST.COM; defaulting to no policy
I20260811 07:01:50.122224 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.121898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50764)
0811 07:01:50.122033 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.122114 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 07:01:50.122429 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:36227: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab.
Entry for principal kudu/127.4.40.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab.
I20260811 07:01:50.163761 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:36227
--webserver_interface=127.4.40.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab
--principal=kudu/127.4.40.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:35089,127.4.40.61:36227,127.4.40.60:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:01:50.171742 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } attempt: 1
I20260811 07:01:50.172983 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.172554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50776)
0811 07:01:50.172750 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.172850 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 07:01:50.174052 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.173754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50778)
0811 07:01:50.173874 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.173936 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 07:01:50.174295 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:36227: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
I20260811 07:01:50.242571 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } attempt: 2
I20260811 07:01:50.243933 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.243349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50794)
0811 07:01:50.243636 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.243746 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 07:01:50.245042 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.244594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50796)
0811 07:01:50.244757 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.244861 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:01:50.245237 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:36227: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
I20260811 07:01:50.336541 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } attempt: 3
I20260811 07:01:50.338128 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.337501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50800)
0811 07:01:50.337750 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.337911 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 07:01:50.339460 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.338915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50812)
0811 07:01:50.339174 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.339272 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149}
W20260811 07:01:50.339689 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:36227: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
I20260811 07:01:50.482946 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } attempt: 4
I20260811 07:01:50.484299 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.483684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50822)
0811 07:01:50.484035 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.484167 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180}
I20260811 07:01:50.485337 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.485011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50838)
0811 07:01:50.485144 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.485219 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 07:01:50.485589 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:36227: Network error: Client connection negotiation failed: client connection to 127.4.40.61:36227: connect: Connection refused (error 111)
W20260811 07:01:50.514384 12480 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:01:50.514786 12480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:50.514914 12480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:50.515048 12480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:50.522300 12480 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:50.522436 12480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:50.522578 12480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:50.522662 12480 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:50.522755 12480 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:50.532598 12480 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:35089,127.4.40.61:36227,127.4.40.60:44823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab
--principal=kudu/127.4.40.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:36227
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.40.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:50.533912 12480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:50.535178 12480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:50.547041 12490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:50.547062 12493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:50.547112 12496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:50.548285 12480 server_base.cc:1034] running on GCE node
I20260811 07:01:50.548996 12480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:50.550248 12480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:50.551532 12480 hybrid_clock.cc:648] HybridClock initialized: now 1786431710551479 us; error 47 us; skew 500 ppm
I20260811 07:01:50.551848 12480 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:50 dist-test-slave-7vjv krb5kdc[12316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:50.555071 12480 init.cc:377] Logged in from keytab as kudu/127.4.40.61@KRBTEST.COM (short username kudu)
I20260811 07:01:50.556468 12480 webserver.cc:460] Webserver started at http://127.4.40.61:39523/ using document root <none> and password file <none>
I20260811 07:01:50.557186 12480 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:50.557315 12480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:50.557634 12480 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:50.560563 12480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data/instance:
uuid: "2e07921c3c21496b9a182661aae0aa53"
format_stamp: "Formatted at 2026-08-11 07:01:50 on dist-test-slave-7vjv"
server_key: "b2513346a61f2458f42717d3e367592f"
server_key_iv: "93d808327c137d88ebcf9c6a43a1f0ee"
server_key_version: "encryptionkey@0"
I20260811 07:01:50.561345 12480 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal/instance:
uuid: "2e07921c3c21496b9a182661aae0aa53"
format_stamp: "Formatted at 2026-08-11 07:01:50 on dist-test-slave-7vjv"
server_key: "b2513346a61f2458f42717d3e367592f"
server_key_iv: "93d808327c137d88ebcf9c6a43a1f0ee"
server_key_version: "encryptionkey@0"
I20260811 07:01:50.566282 12480 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 07:01:50.569694 12513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:50.570940 12480 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 07:01:50.571122 12480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal
uuid: "2e07921c3c21496b9a182661aae0aa53"
format_stamp: "Formatted at 2026-08-11 07:01:50 on dist-test-slave-7vjv"
server_key: "b2513346a61f2458f42717d3e367592f"
server_key_iv: "93d808327c137d88ebcf9c6a43a1f0ee"
server_key_version: "encryptionkey@0"
I20260811 07:01:50.571332 12480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:50.596163 12480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:50.600741 12480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:50.601015 12480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:50.619549 12480 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.61:36227
I20260811 07:01:50.619556 12596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.61:36227 every 8 connection(s)
I20260811 07:01:50.620944 12480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
I20260811 07:01:50.624668 12597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:50.628304 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12480
I20260811 07:01:50.628504 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-1/wal/instance
I20260811 07:01:50.628980 4256 external_mini_cluster.cc:1428] Setting key 987b196c8c350e72de0d3df9c94d7305
I20260811 07:01:50.631172 12597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:50 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:50.652176 12600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.633745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43654 (local address 127.4.40.62:35089)
0811 07:01:50.634293 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:50.634298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:50.635194 (+ 896us) server_negotiation.cc:407] Connection header received
0811 07:01:50.637436 (+ 2242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:50.637463 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:50.637546 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:50.637677 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:50.639248 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.640034 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.641592 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.642033 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.642188 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.646104 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:50.646109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:50.646113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:50.646148 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:50.648801 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.649765 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.649769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.649772 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.649888 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.650430 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.650433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.650435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.650696 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:50.650815 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:50.651670 (+ 855us) server_negotiation.cc:299] Negotiation successful
0811 07:01:50.651825 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":307,"threads_started":1}
I20260811 07:01:50.653590 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.634290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35089 (local address 127.0.0.1:43654)
0811 07:01:50.635034 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.635069 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:50.636961 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:50.637951 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:50.637959 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:50.638476 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:50.639024 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.639041 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.640247 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.640252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.641437 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.641449 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.642282 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.642289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.642529 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.643232 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:50.643253 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:50.645862 (+ 2609us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:50.648942 (+ 3080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.648953 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.648969 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.649648 (+ 679us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.650009 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.650016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.650022 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.650285 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.650832 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:50.650841 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:50.651274 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 07:01:50.651608 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 07:01:50.652191 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":331,"threads_started":1}
I20260811 07:01:50.655313 12597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:50 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.40.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.60@KRBTEST.COM" created.
I20260811 07:01:50.672281 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.656207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50848)
0811 07:01:50.656390 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.656410 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:50.656528 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:50.657658 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:50.657661 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:50.658183 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:50.658403 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.658413 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.660580 (+ 2167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.660584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.661397 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.661407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.662685 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.662689 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.662819 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.663401 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:50.663417 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:50.665843 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:50.669727 (+ 3884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.669730 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.669733 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.670070 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.670533 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.670536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.670538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.670723 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.671337 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:50.671342 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:50.671483 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 07:01:50.671637 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 07:01:50.671839 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73,"mutex_wait_us":2}
I20260811 07:01:50.672544 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.656361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.4.40.61:36227)
0811 07:01:50.657080 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:50.657090 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:50.657123 (+ 33us) server_negotiation.cc:407] Connection header received
0811 07:01:50.657177 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:50.657187 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:50.657485 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:50.657662 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:50.658620 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.660419 (+ 1799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.661541 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.662526 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.662687 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.666079 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:50.666096 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:50.666116 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:50.666157 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:50.669589 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.670254 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.670261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.670263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.670427 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.670840 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.670844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.670846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.671203 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:50.671444 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:50.671848 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 07:01:50.672081 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":534,"thread_start_us":382,"threads_started":1}
I20260811 07:01:50.675145 12597 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:50.676378 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.675774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50136)
0811 07:01:50.675985 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.676185 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 07:01:50.676671 12514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:50.678321 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.677889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50140)
0811 07:01:50.678078 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.678183 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 07:01:50.678560 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab.
Entry for principal kudu/127.4.40.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab.
I20260811 07:01:50.711225 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:44823
--webserver_interface=127.4.40.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab
--principal=kudu/127.4.40.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:37667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:35089,127.4.40.61:36227,127.4.40.60:44823
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:01:50.727892 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 1
I20260811 07:01:50.729135 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.728687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50148)
0811 07:01:50.728888 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.728982 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 07:01:50.730124 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.729779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50162)
0811 07:01:50.729946 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.730023 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 07:01:50.730346 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:50.784868 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } attempt: 5
Aug 11 07:01:50 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 07:01:50.798599 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 2
I20260811 07:01:50.799751 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.799149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50170)
0811 07:01:50.799452 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.799548 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197}
I20260811 07:01:50.800297 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.785645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50860)
0811 07:01:50.785870 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.785892 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:50.786031 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:50.786494 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:50.786497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:50.786895 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:50.787073 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.787081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.788583 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.788587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.789448 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:50.789477 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.790568 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.790572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:50.790711 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.791404 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:50.791422 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:50.793742 (+ 2320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:50.797535 (+ 3793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.797538 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.797541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.797895 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.798297 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:50.798300 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:50.798303 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:50.798423 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:50.799166 (+ 743us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:50.799171 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:50.799401 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 07:01:50.799634 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 07:01:50.799908 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 07:01:50.800812 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.785870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.4.40.61:36227)
0811 07:01:50.786114 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:50.786123 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:50.786148 (+ 25us) server_negotiation.cc:407] Connection header received
0811 07:01:50.786218 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:50.786223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:50.786325 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:50.786453 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:50.787243 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.788433 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.789641 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:50.790425 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:50.790557 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:50.793939 (+ 3382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:50.793945 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:50.793950 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:50.793989 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:50.797390 (+ 3401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.798027 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.798032 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.798036 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.798186 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:50.798530 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:50.798535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:50.798538 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:50.798996 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:50.799178 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:50.800256 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0811 07:01:50.800512 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 07:01:50.801218 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.800468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50176)
0811 07:01:50.800972 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.801063 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397}
W20260811 07:01:50.801507 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:50.801853 12469 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:50.802740 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.802384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50188)
0811 07:01:50.802535 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.802612 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 07:01:50.803602 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.803286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50198)
0811 07:01:50.803424 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.803513 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 07:01:50.803829 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:50.855134 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 1
I20260811 07:01:50.856482 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.855819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50212)
0811 07:01:50.856126 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.856277 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 07:01:50.857537 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.857172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50228)
0811 07:01:50.857322 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.857409 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 07:01:50.857752 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:50.892740 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 3
I20260811 07:01:50.893893 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.893394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50242)
0811 07:01:50.893614 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.893719 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 07:01:50.894994 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.894598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50258)
0811 07:01:50.894828 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.894886 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 07:01:50.895208 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:50.925999 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 2
I20260811 07:01:50.927250 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.926732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50264)
0811 07:01:50.926987 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.927078 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 07:01:50.928283 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:50.927939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50280)
0811 07:01:50.928069 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:50.928185 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:01:50.928541 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:51.034950 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 3
I20260811 07:01:51.036317 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.035743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50290)
0811 07:01:51.035944 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.036045 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 07:01:51.037348 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.037017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50302)
0811 07:01:51.037182 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.037256 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 07:01:51.037616 12469 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
I20260811 07:01:51.038421 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 4
I20260811 07:01:51.039371 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.038931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50304)
0811 07:01:51.039144 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.039215 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 07:01:51.040391 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.040052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50310)
0811 07:01:51.040226 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.040296 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 07:01:51.040581 12597 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44823: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44823: connect: Connection refused (error 111)
W20260811 07:01:51.062659 12618 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:01:51.063019 12618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:51.063139 12618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:51.063270 12618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:51.070441 12618 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:51.070571 12618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:51.070690 12618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:51.070776 12618 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:51.070884 12618 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:51.085951 12618 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:37667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:35089,127.4.40.61:36227,127.4.40.60:44823
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab
--principal=kudu/127.4.40.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:44823
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.40.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:51.087493 12618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:51.089165 12618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:51.100872 12627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:51.101033 12625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:51.100986 12632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:51.101907 12618 server_base.cc:1034] running on GCE node
I20260811 07:01:51.102595 12618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:51.103891 12618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:51.105141 12618 hybrid_clock.cc:648] HybridClock initialized: now 1786431711105101 us; error 43 us; skew 500 ppm
I20260811 07:01:51.105474 12618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431711, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:51.108551 12618 init.cc:377] Logged in from keytab as kudu/127.4.40.60@KRBTEST.COM (short username kudu)
I20260811 07:01:51.109982 12618 webserver.cc:460] Webserver started at http://127.4.40.60:41861/ using document root <none> and password file <none>
I20260811 07:01:51.110658 12618 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:51.110795 12618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:51.111120 12618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:51.114040 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data/instance:
uuid: "1cb85a39e7224f369224abd108b661f5"
format_stamp: "Formatted at 2026-08-11 07:01:51 on dist-test-slave-7vjv"
server_key: "212d49aa297e9d01ea9e656536f13fa6"
server_key_iv: "f1d40e4f2ee6c62afb122c4ece4f09e4"
server_key_version: "encryptionkey@0"
I20260811 07:01:51.114873 12618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal/instance:
uuid: "1cb85a39e7224f369224abd108b661f5"
format_stamp: "Formatted at 2026-08-11 07:01:51 on dist-test-slave-7vjv"
server_key: "212d49aa297e9d01ea9e656536f13fa6"
server_key_iv: "f1d40e4f2ee6c62afb122c4ece4f09e4"
server_key_version: "encryptionkey@0"
I20260811 07:01:51.119961 12618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 07:01:51.123397 12645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:51.124821 12618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 07:01:51.125000 12618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal
uuid: "1cb85a39e7224f369224abd108b661f5"
format_stamp: "Formatted at 2026-08-11 07:01:51 on dist-test-slave-7vjv"
server_key: "212d49aa297e9d01ea9e656536f13fa6"
server_key_iv: "f1d40e4f2ee6c62afb122c4ece4f09e4"
server_key_version: "encryptionkey@0"
I20260811 07:01:51.125221 12618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:51.162241 12618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:51.166267 12618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:51.166558 12618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:51.185793 12618 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.60:44823
I20260811 07:01:51.185817 12733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.60:44823 every 8 connection(s)
I20260811 07:01:51.187139 12618 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
I20260811 07:01:51.190806 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:51.196205 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 12618
I20260811 07:01:51.196403 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786431333861527-4256-0/minicluster-data/master-2/wal/instance
I20260811 07:01:51.196835 4256 external_mini_cluster.cc:1428] Setting key 0b0763800354b72bc0b44f4f1cdb158c
I20260811 07:01:51.197196 12734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431711, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 07:01:51.214848 12469 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 4
I20260811 07:01:51.219053 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.199958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43668 (local address 127.4.40.62:35089)
0811 07:01:51.200861 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.200868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.201073 (+ 205us) server_negotiation.cc:407] Connection header received
0811 07:01:51.202180 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.202184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.202259 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.202387 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.203830 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.204628 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.206396 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.206947 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.207051 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.210923 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.210933 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.210940 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.210995 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.214488 (+ 3493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.215554 (+ 1066us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.215563 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.215568 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.215742 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.216382 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.216386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.216388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.216693 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.216866 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.218409 (+ 1543us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.218637 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":773,"thread_start_us":250,"threads_started":1}
I20260811 07:01:51.219825 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.199991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:35089 (local address 127.0.0.1:43668)
0811 07:01:51.200915 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.200958 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:51.201914 (+ 956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:51.202443 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:51.202450 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:51.203284 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:51.203649 (+ 365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.203666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.204808 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.204814 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.206229 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.206241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.207166 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.207173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.207423 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.208308 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:51.208330 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:51.210714 (+ 2384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:51.214647 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.214657 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.214667 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.215356 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.215897 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.215902 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.215905 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.216220 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.216846 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:51.216854 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:51.217362 (+ 508us) client_negotiation.cc:769] Sending connection context
0811 07:01:51.217707 (+ 345us) client_negotiation.cc:240] Negotiation successful
0811 07:01:51.218272 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"spinlock_wait_cycles":18304,"thread_start_us":499,"threads_started":1}
I20260811 07:01:51.220276 12737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.198682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50316 (local address 127.4.40.60:44823)
0811 07:01:51.199442 (+ 760us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.199451 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.199493 (+ 42us) server_negotiation.cc:407] Connection header received
0811 07:01:51.201914 (+ 2421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.201941 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.202230 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.202433 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.203905 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.205354 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.206726 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.207365 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.207504 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.211980 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.211991 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.212007 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.212047 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.215807 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.216756 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.216767 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.216773 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.216950 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.217349 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.217357 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.217362 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.217824 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.218212 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.219505 (+ 1293us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.219769 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":355,"threads_started":1}
I20260811 07:01:51.221665 12734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431711, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 07:01:51.231673 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.215519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50332)
0811 07:01:51.215799 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.215819 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:51.215924 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:51.216840 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:51.216845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:51.217348 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:51.217553 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.217564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.219234 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.219239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.220535 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.220548 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.221590 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.221596 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.221785 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.222446 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:51.222463 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:51.225162 (+ 2699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:51.228799 (+ 3637us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.228805 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.228810 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.229292 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.229784 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.229789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.229794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.229974 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.230586 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:51.230592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:51.230775 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 07:01:51.230927 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 07:01:51.231242 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 07:01:51.231756 12743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.215742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50332 (local address 127.4.40.60:44823)
0811 07:01:51.216511 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.216519 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.216543 (+ 24us) server_negotiation.cc:407] Connection header received
0811 07:01:51.216588 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.216593 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.216662 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.217386 (+ 724us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.217680 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.219067 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.220722 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.221412 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.221566 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.225378 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.225382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.225385 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.225412 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.228668 (+ 3256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.229457 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.229474 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.229478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.229642 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.230151 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.230157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.230160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.230438 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.230663 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.231048 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.231278 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":632,"thread_start_us":440,"threads_started":1}
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:51.237735 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.222402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:36227 (local address 127.0.0.1:50870)
0811 07:01:51.222545 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.222565 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:51.222666 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:51.223673 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:51.223677 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:51.224098 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:51.224319 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.224334 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.225787 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.225794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.227301 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.227319 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.228016 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.228022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.228276 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.228863 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:51.228879 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:51.231412 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:51.234988 (+ 3576us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.235011 (+ 23us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.235015 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.235517 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.235991 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.235996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.236000 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.236198 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.236825 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:51.236831 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:51.236978 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 07:01:51.237150 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 07:01:51.237406 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 07:01:51.238183 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.223067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50870 (local address 127.4.40.61:36227)
0811 07:01:51.223220 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.223225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.223240 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:01:51.223317 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.223322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.223397 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.223504 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.224558 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.225610 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.227456 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.227852 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.227996 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.231647 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.231654 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.231658 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.231699 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.234652 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.235654 (+ 1002us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.235663 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.235669 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.235844 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.236322 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.236327 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.236329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.236701 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.236857 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.237333 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.237556 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 07:01:51.238258 12469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15: Bootstrap starting.
I20260811 07:01:51.238986 12734 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:51.242532 12469 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:51.243661 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.223472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43684 (local address 127.4.40.62:35089)
0811 07:01:51.224036 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.224040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.224053 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:01:51.224133 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.224140 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.224220 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.224333 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.227510 (+ 3177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.228582 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.230533 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.231200 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.231332 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.236587 (+ 5255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.236593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.236637 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.236677 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.240482 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.241316 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.241320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.241322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.241450 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.241878 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.241883 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.241887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.242170 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.242467 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.243281 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.243452 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448}
I20260811 07:01:51.243852 12469 log.cc:824] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:51.247220 12469 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15: No bootstrap required, opened a new log
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431711, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431709, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 07:01:51.255086 12743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.239883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50340 (local address 127.4.40.60:44823)
0811 07:01:51.240133 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.240139 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.240153 (+ 14us) server_negotiation.cc:407] Connection header received
0811 07:01:51.240192 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.240200 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.240259 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.240359 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.241260 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.242289 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.244353 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.244992 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.245130 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.248786 (+ 3656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.248792 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.248796 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.248830 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.252334 (+ 3504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.253075 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.253080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.253084 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.253254 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.253675 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.253681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.253685 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.253995 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.254130 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.254556 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.254762 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260811 07:01:51.255188 12742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.239675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50340)
0811 07:01:51.239823 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.239841 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:51.239933 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:51.240383 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:51.240387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:51.240882 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:51.241095 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.241108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.242453 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.242459 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.244187 (+ 1728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.244202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.245338 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.245345 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.245533 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.246138 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:51.246177 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:51.248578 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:51.252465 (+ 3887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.252470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.252473 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.252942 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.253394 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.253399 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.253402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.253557 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.254178 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:51.254182 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:51.254333 (+ 151us) client_negotiation.cc:769] Sending connection context
0811 07:01:51.254481 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 07:01:51.254867 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":140}
I20260811 07:01:51.257640 12469 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.258039 12469 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f91b479b6b047c3b894cebbaa02bc15, State: Initialized, Role: FOLLOWER
I20260811 07:01:51.258534 12469 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.259516 12745 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.259931 12745 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.260221 12734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: Bootstrap starting.
I20260811 07:01:51.260838 12469 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:51.261709 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.245642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50882 (local address 127.4.40.61:36227)
0811 07:01:51.245841 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.245844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.245857 (+ 13us) server_negotiation.cc:407] Connection header received
0811 07:01:51.245893 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.245896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.245946 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.246050 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.247402 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.248065 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.249315 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.249894 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.250042 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.254336 (+ 4294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.254343 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.254347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.254384 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.258246 (+ 3862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.259123 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.259128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.259131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.259301 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.259840 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.259845 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.259848 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.260217 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.260481 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.261131 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.261370 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 07:01:51.264003 12734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:51.265411 12734 log.cc:824] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:51.267407 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:51.268738 12734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: No bootstrap required, opened a new log
I20260811 07:01:51.272291 12421 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "7f91b479b6b047c3b894cebbaa02bc15"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43684
I20260811 07:01:51.272779 12421 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 07:01:51.273049 12421 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:51.276750 12421 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.277808 12734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.278138 12734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb85a39e7224f369224abd108b661f5, State: Initialized, Role: FOLLOWER
I20260811 07:01:51.278644 12421 leader_election.cc:288] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [CANDIDATE]: Term 1 election: Requested vote from peers 2e07921c3c21496b9a182661aae0aa53 (127.4.40.61:36227), 1cb85a39e7224f369224abd108b661f5 (127.4.40.60:44823)
I20260811 07:01:51.278591 12734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.279870 12772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.280709 12772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.281036 12734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:51.281155 12690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f91b479b6b047c3b894cebbaa02bc15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1cb85a39e7224f369224abd108b661f5"
I20260811 07:01:51.281631 12690 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:51.286011 12770 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 07:01:51.287323 12690 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f91b479b6b047c3b894cebbaa02bc15 in term 1.
I20260811 07:01:51.288054 12372 leader_election.cc:302] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cb85a39e7224f369224abd108b661f5, 7f91b479b6b047c3b894cebbaa02bc15; no voters:
I20260811 07:01:51.288515 12745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:51.289645 12745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [term 1 LEADER]: Becoming Leader. State: Replica: 7f91b479b6b047c3b894cebbaa02bc15, State: Running, Role: LEADER
I20260811 07:01:51.290249 12745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
W20260811 07:01:51.303244 12803 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:01:51.303468 12803 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:01:51.292212 12344 master_runner.cc:186] Error getting master registration for 127.4.40.61:36227: OK, instance_id { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" instance_seqno: 1786431710614301 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:01:51.307030 12344 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:01:51.308732 12790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f91b479b6b047c3b894cebbaa02bc15. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.309130 12790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:51.310129 12805 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 07:01:51.311796 12379 tablet.cc:2369] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:01:51.323979 12805 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:51.339785 12597 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } attempt: 5
I20260811 07:01:51.348910 12690 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [term 1 FOLLOWER]: Refusing update from remote peer 7f91b479b6b047c3b894cebbaa02bc15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:51.349936 12790 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 11 07:01:51 dist-test-slave-7vjv krb5kdc[12316](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431710, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 07:01:51.378739 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.332466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50896 (local address 127.4.40.61:36227)
0811 07:01:51.332644 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.332650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.356692 (+ 24042us) server_negotiation.cc:407] Connection header received
0811 07:01:51.356860 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.356866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.356938 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.357029 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.359121 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.360322 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.363119 (+ 2797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.363792 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.363886 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.369761 (+ 5875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.369768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.369772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.369810 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.373273 (+ 3463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.374082 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.374088 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.374092 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.374229 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.374719 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.374724 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.374727 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.375084 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.375379 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.376588 (+ 1209us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.376813 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:01:51.383097 12604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.340433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44823 (local address 127.0.0.1:50352)
0811 07:01:51.340854 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:51.340877 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:51.341010 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:51.341998 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:51.342003 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:51.342573 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:51.342781 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.342794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.344679 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.344685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.346156 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:51.346171 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.353488 (+ 7317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.353494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:51.353713 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.354617 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:51.354645 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:51.367406 (+ 12761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:51.376858 (+ 9452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.376864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.376867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.377618 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.378866 (+ 1248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:51.378870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:51.378874 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:51.379175 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:51.381090 (+ 1915us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:51.381095 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:51.381362 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 07:01:51.381655 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 07:01:51.382063 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260811 07:01:51.390136 12743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.341259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50352 (local address 127.4.40.60:44823)
0811 07:01:51.341467 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.341473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.341493 (+ 20us) server_negotiation.cc:407] Connection header received
0811 07:01:51.341581 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.341599 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.341699 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.341801 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.343301 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.344432 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.346994 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.347702 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.347829 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.367682 (+ 19853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.367689 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.367694 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.367734 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.376643 (+ 8909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.377931 (+ 1288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.377939 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.377945 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.378472 (+ 527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.379968 (+ 1496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.379974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.379978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.380871 (+ 893us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.382777 (+ 1906us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.383723 (+ 946us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.384741 (+ 1018us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:01:51.390726 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.326964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43700 (local address 127.4.40.62:35089)
0811 07:01:51.327135 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.327142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.342979 (+ 15837us) server_negotiation.cc:407] Connection header received
0811 07:01:51.344086 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.344106 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.344180 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.344251 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.350338 (+ 6087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.351577 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.355025 (+ 3448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.355712 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.355814 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.362476 (+ 6662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.362484 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.362494 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.362543 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.366264 (+ 3721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.382018 (+ 15754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.382024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.382028 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.382244 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.386538 (+ 4294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.386545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.386554 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.386947 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.387238 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.390045 (+ 2807us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.390294 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 07:01:51.397099 12597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: Bootstrap starting.
I20260811 07:01:51.398617 12737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.357362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50356 (local address 127.4.40.60:44823)
0811 07:01:51.357533 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.357538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.376440 (+ 18902us) server_negotiation.cc:407] Connection header received
0811 07:01:51.377170 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.377175 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.377251 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.377317 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.378641 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.379734 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.382810 (+ 3076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.383515 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.383619 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.391566 (+ 7947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:51.391572 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:51.391577 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:51.391616 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:51.395309 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.396238 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.396243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.396247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.396405 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:51.396869 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:51.396879 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:51.396882 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:51.397244 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:51.397460 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.398052 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.398268 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 07:01:51.401473 12597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:51.402746 12597 log.cc:824] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:51.327639 12618 master_runner.cc:186] Error getting master registration for 127.4.40.61:36227: OK, instance_id { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" instance_seqno: 1786431710614301 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:01:51.404769 12618 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:01:51.409710 12597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: No bootstrap required, opened a new log
W20260811 07:01:51.418648 12657 tablet.cc:2369] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:01:51.429899 12597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.430308 12597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e07921c3c21496b9a182661aae0aa53, State: Initialized, Role: FOLLOWER
I20260811 07:01:51.430842 12597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } }
I20260811 07:01:51.432150 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.432861 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.432812 12772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f91b479b6b047c3b894cebbaa02bc15. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.433092 12772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.436256 12597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:51.436847 12813 mvcc.cc:205] Tried to move back new op lower bound from 7317224289670815744 to 7317224289450070016. Current Snapshot: MvccSnapshot[applied={T|T < 7317224289670815744}]
I20260811 07:01:51.437431 12549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:51.440303 12551 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7f91b479b6b047c3b894cebbaa02bc15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2e07921c3c21496b9a182661aae0aa53"
I20260811 07:01:51.447566 12772 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.447816 12772 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.448966 12549 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [term 1 FOLLOWER]: Refusing update from remote peer 7f91b479b6b047c3b894cebbaa02bc15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:51.452315 12805 catalog_manager.cc:1347] Generated new cluster ID: 0d8cbf890169429991f2571aec67d7cd
I20260811 07:01:51.452417 12805 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:51.456626 12551 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f91b479b6b047c3b894cebbaa02bc15 in term 1.
I20260811 07:01:51.461012 12745 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:51.465440 12790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.465854 12790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:51.509058 12805 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:51.511686 12805 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:51.511763 12820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f91b479b6b047c3b894cebbaa02bc15. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.512120 12820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.514402 12831 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.514846 12831 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15 [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:51.516419 12837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7f91b479b6b047c3b894cebbaa02bc15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f91b479b6b047c3b894cebbaa02bc15" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 35089 } } peers { permanent_uuid: "2e07921c3c21496b9a182661aae0aa53" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 36227 } } peers { permanent_uuid: "1cb85a39e7224f369224abd108b661f5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44823 } } }
I20260811 07:01:51.516825 12837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:51.537597 12805 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f91b479b6b047c3b894cebbaa02bc15: Generated new TSK 0
I20260811 07:01:51.539027 12805 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:51.559417 12854 catalog_manager.cc:1259] Loaded cluster ID: 0d8cbf890169429991f2571aec67d7cd
I20260811 07:01:51.559643 12854 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: loading cluster ID for follower catalog manager: success
I20260811 07:01:51.563386 12854 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: acquiring CA information for follower catalog manager: success
I20260811 07:01:51.577589 12854 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2e07921c3c21496b9a182661aae0aa53: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 07:01:51.714192 12738 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:51.686122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43708 (local address 127.4.40.62:35089)
0811 07:01:51.686353 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.686360 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.708245 (+ 21885us) server_negotiation.cc:407] Connection header received
0811 07:01:51.711154 (+ 2909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.711162 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.711257 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.711339 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.713901 (+ 2562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43708: BlockingRecv error: recv got EOF from 127.0.0.1:43708 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 07:01:51.718631 12610 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:51.692734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50900 (local address 127.4.40.61:36227)
0811 07:01:51.692930 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.692936 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.715462 (+ 22526us) server_negotiation.cc:407] Connection header received
0811 07:01:51.716866 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.716873 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.716968 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.717052 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.718347 (+ 1295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50900: BlockingRecv error: recv got EOF from 127.0.0.1:50900 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 07:01:51.720850 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:51.708864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.4.40.60:44823)
0811 07:01:51.709120 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.709129 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.716142 (+ 7013us) server_negotiation.cc:407] Connection header received
0811 07:01:51.719276 (+ 3134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.719283 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.719386 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.719464 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.720559 (+ 1095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50366: BlockingRecv error: recv got EOF from 127.0.0.1:50366 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 07:01:51.772567 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.751256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.4.40.61:36227)
0811 07:01:51.751472 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.751479 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.762656 (+ 11177us) server_negotiation.cc:407] Connection header received
0811 07:01:51.763466 (+ 810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.763474 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.763736 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.763928 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:51.765013 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.766400 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.769173 (+ 2773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.769925 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.770030 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.770896 (+ 866us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:51.771634 (+ 738us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:51.771778 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.772022 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.772153 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 07:01:51.820598 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:51.806998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50380 (local address 127.4.40.60:44823)
0811 07:01:51.807207 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.807213 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.817919 (+ 10706us) server_negotiation.cc:407] Connection header received
0811 07:01:51.818476 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.818485 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.818655 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.818744 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.820292 (+ 1548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50380: BlockingRecv error: recv got EOF from 127.0.0.1:50380 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:01:51.876575 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.857352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50916 (local address 127.4.40.61:36227)
0811 07:01:51.857602 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.857610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.868347 (+ 10737us) server_negotiation.cc:407] Connection header received
0811 07:01:51.868891 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.868898 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.869008 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.869090 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:51.869901 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.871274 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.873673 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.874546 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.874669 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.875521 (+ 852us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:51.875691 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:51.875759 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.876117 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.876211 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 07:01:51.929520 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:51.911914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.4.40.60:44823)
0811 07:01:51.912139 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.912149 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.916664 (+ 4515us) server_negotiation.cc:407] Connection header received
0811 07:01:51.916923 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.916932 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.917077 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.917211 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.929249 (+ 12038us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50392: BlockingRecv error: recv got EOF from 127.0.0.1:50392 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 07:01:51.973299 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.965700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.4.40.61:36227)
0811 07:01:51.965940 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.965947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.966593 (+ 646us) server_negotiation.cc:407] Connection header received
0811 07:01:51.966804 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.966810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.966904 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.967022 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:51.967618 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.968754 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.970905 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.971638 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:51.971805 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:51.972354 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:51.972504 (+ 150us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:51.972629 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:51.972856 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 07:01:51.972941 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 07:01:51.983982 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:51.981995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50404 (local address 127.4.40.60:44823)
0811 07:01:51.982238 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.982244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.982272 (+ 28us) server_negotiation.cc:407] Connection header received
0811 07:01:51.982583 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.982590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.982713 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.982832 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:51.983738 (+ 906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50404: BlockingRecv error: recv got EOF from 127.0.0.1:50404 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 07:01:52.003136 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:51.992278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50936 (local address 127.4.40.61:36227)
0811 07:01:51.992534 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:51.992540 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:51.996725 (+ 4185us) server_negotiation.cc:407] Connection header received
0811 07:01:51.996909 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:51.996915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:51.997047 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:51.997179 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:51.997638 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:51.998815 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.000806 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.001570 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.001713 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.002226 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.002382 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.002488 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.002723 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.002805 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
W20260811 07:01:52.016834 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:52.014288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.4.40.60:44823)
0811 07:01:52.014606 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.014612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.014633 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:01:52.014893 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.014899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.015021 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.015142 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:52.016390 (+ 1248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50410: BlockingRecv error: recv got EOF from 127.0.0.1:50410 (error 108)
Metrics: {"server-negotiator.queue_time_us":180}
I20260811 07:01:52.037014 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.029267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.4.40.61:36227)
0811 07:01:52.029511 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.029518 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.030074 (+ 556us) server_negotiation.cc:407] Connection header received
0811 07:01:52.030228 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.030234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.030320 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.030417 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.030965 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.032196 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.034233 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.034871 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.035018 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.035821 (+ 803us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.035972 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.036230 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.036600 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.036692 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
W20260811 07:01:52.047755 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:52.045252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50414 (local address 127.4.40.60:44823)
0811 07:01:52.045478 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.045488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.045796 (+ 308us) server_negotiation.cc:407] Connection header received
0811 07:01:52.046094 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.046101 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.046196 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.046287 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:52.047507 (+ 1220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50414: BlockingRecv error: recv got EOF from 127.0.0.1:50414 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 07:01:52.076145 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.068457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50954 (local address 127.4.40.61:36227)
0811 07:01:52.068664 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.068672 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.069193 (+ 521us) server_negotiation.cc:407] Connection header received
0811 07:01:52.069422 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.069429 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.069513 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.069586 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.070277 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.071634 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.073733 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.074478 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.074611 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.075157 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.075299 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.075424 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.075723 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.075827 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 07:01:52.086198 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:52.083725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50420 (local address 127.4.40.60:44823)
0811 07:01:52.083986 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.083996 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.084773 (+ 777us) server_negotiation.cc:407] Connection header received
0811 07:01:52.084918 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.084924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.085034 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.085152 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:52.085942 (+ 790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50420: BlockingRecv error: recv got EOF from 127.0.0.1:50420 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 07:01:52.126683 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.121166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50964 (local address 127.4.40.61:36227)
0811 07:01:52.121359 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.121365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.121876 (+ 511us) server_negotiation.cc:407] Connection header received
0811 07:01:52.122127 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.122134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.122221 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.122327 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.122809 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.123510 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.124889 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.125328 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.125469 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.125900 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.125999 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.126109 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.126261 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.126353 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 07:01:52.134750 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:52.132392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50434 (local address 127.4.40.60:44823)
0811 07:01:52.132617 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.132627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.133031 (+ 404us) server_negotiation.cc:407] Connection header received
0811 07:01:52.133173 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.133178 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.133287 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.133371 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:52.134586 (+ 1215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50434: BlockingRecv error: recv got EOF from 127.0.0.1:50434 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 07:01:52.207464 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.202358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50966 (local address 127.4.40.61:36227)
0811 07:01:52.202643 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.202650 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.203029 (+ 379us) server_negotiation.cc:407] Connection header received
0811 07:01:52.203251 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.203255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.203326 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.203410 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.203784 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.204614 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.205924 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.206298 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.206399 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.206780 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.206884 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.206950 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.207073 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.207135 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 07:01:52.215178 12737 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:52.213151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.4.40.60:44823)
0811 07:01:52.213351 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.213358 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.213761 (+ 403us) server_negotiation.cc:407] Connection header received
0811 07:01:52.213972 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.213976 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.214058 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.214169 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:52.214954 (+ 785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50440: BlockingRecv error: recv got EOF from 127.0.0.1:50440 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 07:01:52.305651 12803 catalog_manager.cc:1259] Loaded cluster ID: 0d8cbf890169429991f2571aec67d7cd
I20260811 07:01:52.305760 12803 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: loading cluster ID for follower catalog manager: success
I20260811 07:01:52.308261 12803 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: acquiring CA information for follower catalog manager: success
I20260811 07:01:52.309947 12803 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1cb85a39e7224f369224abd108b661f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:01:52.352478 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.347249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50980 (local address 127.4.40.61:36227)
0811 07:01:52.347488 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.347495 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.347784 (+ 289us) server_negotiation.cc:407] Connection header received
0811 07:01:52.348002 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.348009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.348077 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.348189 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.348677 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.349406 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.350718 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.351161 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.351304 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.351745 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.351840 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.351932 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.352083 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.352201 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 07:01:52.363281 12737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.357873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.4.40.60:44823)
0811 07:01:52.358103 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.358109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.358331 (+ 222us) server_negotiation.cc:407] Connection header received
0811 07:01:52.358471 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.358476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.358629 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.358724 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.359062 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.359852 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.361265 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.361687 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.361775 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.362247 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.362742 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.362856 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.362922 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.362986 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 07:01:52.376953 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.368969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43722 (local address 127.4.40.62:35089)
0811 07:01:52.369483 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.369491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.369947 (+ 456us) server_negotiation.cc:407] Connection header received
0811 07:01:52.370202 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.370210 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.370425 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.370637 (+ 212us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.371243 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.372173 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.374046 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.374694 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.374850 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.375730 (+ 880us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.376199 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.376319 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.376536 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.376627 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":236,"threads_started":1}
I20260811 07:01:52.378407 12737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.370186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50446 (local address 127.4.40.60:44823)
0811 07:01:52.370360 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.370365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.370724 (+ 359us) server_negotiation.cc:407] Connection header received
0811 07:01:52.370954 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.370960 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.371056 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.371153 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.371743 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.372584 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.375639 (+ 3055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.376328 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.376419 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.377158 (+ 739us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.377324 (+ 166us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.377406 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.378020 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.378105 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 07:01:52.379401 12610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:52.369170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.4.40.61:36227)
0811 07:01:52.369336 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:52.369343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:52.370860 (+ 1517us) server_negotiation.cc:407] Connection header received
0811 07:01:52.371034 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:52.371039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:52.371113 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:52.371212 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:52.372995 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.374694 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.377052 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:52.377593 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:52.377688 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:52.378685 (+ 997us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:52.378827 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:52.378925 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:52.379078 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:01:52.379154 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 07:01:52.382349 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12344
I20260811 07:01:52.383594 12454 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:52.494240 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12344
I20260811 07:01:52.527702 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12480
I20260811 07:01:52.528934 12588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:52.639703 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12480
I20260811 07:01:52.670399 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 12618
I20260811 07:01:52.671725 12727 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:52.780663 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 12618
2026-08-11T07:01:52Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4901 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 07:01:52 dist-test-slave-7vjv krb5kdc[13065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 07:01:52 dist-test-slave-7vjv krb5kdc[13065](info): set up 2 sockets
Aug 11 07:01:52 dist-test-slave-7vjv krb5kdc[13065](info): commencing operation
krb5kdc: starting...
W20260811 07:01:54.194953 4256 mini_kdc.cc:121] Time spent starting KDC: real 1.362s user 0.003s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/test-user.keytab.
Aug 11 07:01:54 dist-test-slave-7vjv krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T07:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T07:01:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.40.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
Entry for principal kudu/127.4.40.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab.
I20260811 07:01:54.525537 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43037
--webserver_interface=127.4.40.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:43037,127.4.40.61:44453,127.4.40.60:44825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 07:01:54.884486 13094 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:01:54.884797 13094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:54.884917 13094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:54.885046 13094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:54.892351 13094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:54.892452 13094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:54.892656 13094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:54.892740 13094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:54.892823 13094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:54.902153 13094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:43037,127.4.40.61:44453,127.4.40.60:44825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.62.keytab
--principal=kudu/127.4.40.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.62:43037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.40.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:54.903414 13094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:54.904654 13094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:54.915930 13113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:54.915858 13106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:54.915938 13107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:54.917112 13094 server_base.cc:1034] running on GCE node
I20260811 07:01:54.917852 13094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:54.919212 13094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:54.920470 13094 hybrid_clock.cc:648] HybridClock initialized: now 1786431714920407 us; error 33 us; skew 500 ppm
I20260811 07:01:54.920817 13094 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:54 dist-test-slave-7vjv krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:54.923840 13094 init.cc:377] Logged in from keytab as kudu/127.4.40.62@KRBTEST.COM (short username kudu)
I20260811 07:01:54.925420 13094 webserver.cc:460] Webserver started at http://127.4.40.62:33729/ using document root <none> and password file <none>
I20260811 07:01:54.926115 13094 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:54.926252 13094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:54.926586 13094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:54.929594 13094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data/instance:
uuid: "996a92e568ba4a07a45af565cdffc03e"
format_stamp: "Formatted at 2026-08-11 07:01:54 on dist-test-slave-7vjv"
server_key: "675e0a33cc190c8caade7cc89cffd1cb"
server_key_iv: "3f5120526705045b6b5b56655fda9f48"
server_key_version: "encryptionkey@0"
I20260811 07:01:54.930379 13094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal/instance:
uuid: "996a92e568ba4a07a45af565cdffc03e"
format_stamp: "Formatted at 2026-08-11 07:01:54 on dist-test-slave-7vjv"
server_key: "675e0a33cc190c8caade7cc89cffd1cb"
server_key_iv: "3f5120526705045b6b5b56655fda9f48"
server_key_version: "encryptionkey@0"
I20260811 07:01:54.935382 13094 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 07:01:54.938685 13127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:54.939983 13094 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 07:01:54.940228 13094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal
uuid: "996a92e568ba4a07a45af565cdffc03e"
format_stamp: "Formatted at 2026-08-11 07:01:54 on dist-test-slave-7vjv"
server_key: "675e0a33cc190c8caade7cc89cffd1cb"
server_key_iv: "3f5120526705045b6b5b56655fda9f48"
server_key_version: "encryptionkey@0"
I20260811 07:01:54.940448 13094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:54.952920 13094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:54.956568 13094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:54.956816 13094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:54.975055 13094 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.62:43037
I20260811 07:01:54.975086 13218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.62:43037 every 8 connection(s)
I20260811 07:01:54.976446 13094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/data/info.pb
I20260811 07:01:54.980032 13219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:54.980118 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13094
I20260811 07:01:54.980315 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-0/wal/instance
I20260811 07:01:54.980988 4256 external_mini_cluster.cc:1428] Setting key 4d742019e63326a680f456e2b6d5fbe1
I20260811 07:01:54.989881 13219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:55 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:55.014360 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:54.993482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43037 (local address 127.0.0.1:49444)
0811 07:01:54.994315 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:54.994376 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:54.996584 (+ 2208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:54.997551 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:54.997556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:54.998215 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:54.998717 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:54.998743 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.000339 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.000347 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.002315 (+ 1968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.002333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.003284 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.003297 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.003589 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.004344 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:55.004377 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:55.007041 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:55.010148 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.010160 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.010171 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.010546 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.010970 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.010974 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.010978 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.011145 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.011856 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:55.011864 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:55.012074 (+ 210us) client_negotiation.cc:769] Sending connection context
0811 07:01:55.012390 (+ 316us) client_negotiation.cc:240] Negotiation successful
0811 07:01:55.013003 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":154,"thread_start_us":363,"threads_started":1}
I20260811 07:01:55.014343 13227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:54.994804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49444 (local address 127.4.40.62:43037)
0811 07:01:54.995642 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:54.995652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:54.995710 (+ 58us) server_negotiation.cc:407] Connection header received
0811 07:01:54.996982 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:54.996991 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:54.997281 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:54.997514 (+ 233us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:54.998933 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.000186 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.002454 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.003020 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.003144 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.007273 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:55.007280 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:55.007290 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:55.007321 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:55.010024 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.010671 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.010684 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.010686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.010853 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.011253 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.011261 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.011264 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.011711 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:55.011918 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:55.012510 (+ 592us) server_negotiation.cc:299] Negotiation successful
0811 07:01:55.012888 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":627,"thread_start_us":469,"threads_started":1}
I20260811 07:01:55.017521 13219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.40.61@KRBTEST.COM; defaulting to no policy
I20260811 07:01:55.018514 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.018142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38484)
0811 07:01:55.018300 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.018406 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 07:01:55.018767 13133 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
I20260811 07:01:55.020298 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.019859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38494)
0811 07:01:55.020001 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.020082 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:01:55.020500 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:44453: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab.
Entry for principal kudu/127.4.40.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab.
I20260811 07:01:55.059279 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:44453
--webserver_interface=127.4.40.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab
--principal=kudu/127.4.40.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:43037,127.4.40.61:44453,127.4.40.60:44825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:01:55.069821 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } attempt: 1
I20260811 07:01:55.070986 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.070519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38510)
0811 07:01:55.070715 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.070828 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 07:01:55.071970 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.071607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38520)
0811 07:01:55.071752 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.071825 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 07:01:55.072196 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:44453: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
I20260811 07:01:55.140501 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } attempt: 2
I20260811 07:01:55.141744 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.141263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38522)
0811 07:01:55.141507 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.141613 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 07:01:55.142813 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.142503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38532)
0811 07:01:55.142624 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.142696 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 07:01:55.143071 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:44453: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
I20260811 07:01:55.234423 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } attempt: 3
I20260811 07:01:55.235541 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.235109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38536)
0811 07:01:55.235311 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.235407 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 07:01:55.236793 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.236220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38548)
0811 07:01:55.236387 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.236452 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 07:01:55.237056 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:44453: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
I20260811 07:01:55.380314 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } attempt: 4
I20260811 07:01:55.381692 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.381139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38562)
0811 07:01:55.381470 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.381561 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260811 07:01:55.382836 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.382525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38574)
0811 07:01:55.382654 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.382720 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 07:01:55.383114 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.61:44453: Network error: Client connection negotiation failed: client connection to 127.4.40.61:44453: connect: Connection refused (error 111)
W20260811 07:01:55.409322 13233 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:01:55.409667 13233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:55.409788 13233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:55.409929 13233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:55.417009 13233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:55.417124 13233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:55.417223 13233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:55.417313 13233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:55.417413 13233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:55.426725 13233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:43037,127.4.40.61:44453,127.4.40.60:44825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.61.keytab
--principal=kudu/127.4.40.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.61:44453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.40.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:55.428021 13233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:55.429316 13233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:55.441097 13241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:55.441190 13239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:55.441182 13248 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:55.442056 13233 server_base.cc:1034] running on GCE node
I20260811 07:01:55.442764 13233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:55.444151 13233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:55.445391 13233 hybrid_clock.cc:648] HybridClock initialized: now 1786431715445316 us; error 39 us; skew 500 ppm
I20260811 07:01:55.445722 13233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:55 dist-test-slave-7vjv krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431715, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:55.453668 13233 init.cc:377] Logged in from keytab as kudu/127.4.40.61@KRBTEST.COM (short username kudu)
I20260811 07:01:55.455142 13233 webserver.cc:460] Webserver started at http://127.4.40.61:37507/ using document root <none> and password file <none>
I20260811 07:01:55.455804 13233 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:55.455917 13233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:55.456308 13233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:55.459164 13233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data/instance:
uuid: "5876f1d15f48458490070eb1e9a1f7b6"
format_stamp: "Formatted at 2026-08-11 07:01:55 on dist-test-slave-7vjv"
server_key: "66f155ae4874f87c404e91e4b9889e46"
server_key_iv: "288ce876ade0e81300c4e54ec5afc21f"
server_key_version: "encryptionkey@0"
I20260811 07:01:55.459971 13233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal/instance:
uuid: "5876f1d15f48458490070eb1e9a1f7b6"
format_stamp: "Formatted at 2026-08-11 07:01:55 on dist-test-slave-7vjv"
server_key: "66f155ae4874f87c404e91e4b9889e46"
server_key_iv: "288ce876ade0e81300c4e54ec5afc21f"
server_key_version: "encryptionkey@0"
I20260811 07:01:55.464891 13233 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 07:01:55.468304 13259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:55.469540 13233 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 07:01:55.469714 13233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal
uuid: "5876f1d15f48458490070eb1e9a1f7b6"
format_stamp: "Formatted at 2026-08-11 07:01:55 on dist-test-slave-7vjv"
server_key: "66f155ae4874f87c404e91e4b9889e46"
server_key_iv: "288ce876ade0e81300c4e54ec5afc21f"
server_key_version: "encryptionkey@0"
I20260811 07:01:55.469920 13233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:55.502374 13233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:55.506353 13233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:55.506649 13233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:55.525764 13233 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.61:44453
I20260811 07:01:55.525785 13352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.61:44453 every 8 connection(s)
I20260811 07:01:55.527124 13233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/data/info.pb
I20260811 07:01:55.530838 13353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:55.533156 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13233
I20260811 07:01:55.533355 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-1/wal/instance
I20260811 07:01:55.533814 4256 external_mini_cluster.cc:1428] Setting key 4cdb7f84625ed2566a64bbce93a2b46c
I20260811 07:01:55.537568 13353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:55 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431715, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:55.561353 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.540036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49450 (local address 127.4.40.62:43037)
0811 07:01:55.540553 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:55.540560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:55.541627 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 07:01:55.544429 (+ 2802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:55.544434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:55.544499 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:55.544628 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:55.546593 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.547237 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.549551 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.549955 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.550061 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.553951 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:55.553960 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:55.553976 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:55.554043 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:55.557798 (+ 3755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.558550 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.558558 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.558564 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.558741 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.559144 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.559151 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.559157 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.559677 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:55.559834 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:55.560562 (+ 728us) server_negotiation.cc:299] Negotiation successful
0811 07:01:55.560819 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":262,"threads_started":1}
I20260811 07:01:55.561852 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.540629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43037 (local address 127.0.0.1:49450)
0811 07:01:55.541437 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.541500 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:55.544162 (+ 2662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:55.544864 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:55.544876 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:55.545666 (+ 790us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:55.546407 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.546432 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.547390 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.547398 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.549394 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.549412 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.550192 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.550201 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.550540 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.551301 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:55.551323 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:55.553728 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:55.557940 (+ 4212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.557947 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.557958 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.558411 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.558847 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.558850 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.558853 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.559024 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.559799 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:55.559805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:55.560187 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 07:01:55.560418 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 07:01:55.560773 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":317,"threads_started":1}
I20260811 07:01:55.563560 13353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:55 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431715, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.40.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.40.60@KRBTEST.COM" created.
I20260811 07:01:55.579854 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.564702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.4.40.61:44453)
0811 07:01:55.565296 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:55.565301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:55.565324 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:01:55.565364 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:55.565369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:55.565514 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:55.565622 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:55.566590 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.567567 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.569444 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.569929 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.570029 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.573626 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:55.573637 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:55.573648 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:55.573675 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:55.576850 (+ 3175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.577697 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.577702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.577705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.577817 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.578281 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.578284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.578286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.578590 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:55.578736 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:55.579203 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 07:01:55.579551 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":461,"thread_start_us":340,"threads_started":1}
I20260811 07:01:55.579933 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.564377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38590)
0811 07:01:55.564541 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.564567 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:55.564639 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:55.565680 (+ 1041us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:55.565688 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:55.566215 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:55.566418 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.566431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.567736 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.567744 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.569298 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.569316 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.570082 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.570091 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.570293 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.570883 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:55.570912 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:55.573445 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:55.576984 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.576992 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.576998 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.577552 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.577926 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.577933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.577938 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.578155 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.578733 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:55.578740 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:55.578912 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 07:01:55.579078 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 07:01:55.579424 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:01:55.582293 13353 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:55.583472 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.582922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59484)
0811 07:01:55.583112 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.583284 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 07:01:55.583745 13265 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.585484 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.585034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59494)
0811 07:01:55.585209 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.585344 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:01:55.585719 13353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.40.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab.
Entry for principal kudu/127.4.40.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab.
I20260811 07:01:55.616918 4256 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
/tmp/dist-test-taskCgIwdj/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:44825
--webserver_interface=127.4.40.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab
--principal=kudu/127.4.40.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.40.20:42541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.40.62:43037,127.4.40.61:44453,127.4.40.60:44825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 07:01:55.635080 13353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 1
I20260811 07:01:55.636368 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.635862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59498)
0811 07:01:55.636069 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.636205 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 07:01:55.637323 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.636994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59514)
0811 07:01:55.637155 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.637223 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 07:01:55.637530 13353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.682430 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } attempt: 5
Aug 11 07:01:55 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 07:01:55.696910 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.683196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38604)
0811 07:01:55.683434 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.683457 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:55.683603 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:55.684026 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:55.684029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:55.684455 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:55.684638 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.684647 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.686054 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.686058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.686948 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:55.686959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.688000 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.688005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:55.688218 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.688848 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:55.688866 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:55.690869 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:55.694474 (+ 3605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.694477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.694480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.694849 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.695228 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:55.695231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:55.695233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:55.695355 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:55.695989 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:55.695992 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:55.696166 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 07:01:55.696302 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 07:01:55.696635 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 07:01:55.697019 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.683445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38604 (local address 127.4.40.61:44453)
0811 07:01:55.683693 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:55.683699 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:55.683722 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:01:55.683801 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:55.683806 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:55.683880 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:55.683982 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:55.684819 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.685897 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.687096 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:55.687755 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:55.687895 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:55.691071 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:55.691079 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:55.691083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:55.691121 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:55.694351 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.694965 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.694971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.694974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.695120 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:55.695474 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:55.695481 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:55.695486 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:55.695868 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:55.696016 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:55.696495 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 07:01:55.696728 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 07:01:55.697916 13219 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:55.698925 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.698445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59516)
0811 07:01:55.698640 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.698731 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 07:01:55.700026 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.699600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59522)
0811 07:01:55.699792 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.699902 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 07:01:55.700268 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.705717 13353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 2
I20260811 07:01:55.706550 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.706173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59530)
0811 07:01:55.706333 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.706414 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 07:01:55.707440 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.707126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59546)
0811 07:01:55.707263 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.707334 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 07:01:55.707653 13353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.751525 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 1
I20260811 07:01:55.752967 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.752325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59552)
0811 07:01:55.752589 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.752727 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 07:01:55.754196 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.753758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59556)
0811 07:01:55.753940 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.754040 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 07:01:55.754449 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.798909 13353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 3
I20260811 07:01:55.800216 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.799642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59572)
0811 07:01:55.799882 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.799970 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 07:01:55.801468 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.800901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59576)
0811 07:01:55.801190 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.801296 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171}
W20260811 07:01:55.801734 13353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.822721 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 2
I20260811 07:01:55.824069 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.823424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59590)
0811 07:01:55.823703 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.823830 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 07:01:55.825277 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.824826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59600)
0811 07:01:55.825013 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.825114 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 07:01:55.825543 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.931810 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 3
I20260811 07:01:55.933118 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.932538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59602)
0811 07:01:55.932794 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.932911 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 07:01:55.934278 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.933822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59614)
0811 07:01:55.934014 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.934129 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 07:01:55.934510 13219 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
I20260811 07:01:55.944977 13353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 4
I20260811 07:01:55.946164 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.945642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59626)
0811 07:01:55.945876 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.945984 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 07:01:55.947347 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:55.946904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59640)
0811 07:01:55.947067 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:55.947169 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 07:01:55.947603 13353 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.40.60:44825: Network error: Client connection negotiation failed: client connection to 127.4.40.60:44825: connect: Connection refused (error 111)
W20260811 07:01:55.969242 13365 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 07:01:55.969513 13365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 07:01:55.969583 13365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 07:01:55.969653 13365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 07:01:55.977279 13365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 07:01:55.977361 13365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 07:01:55.977473 13365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 07:01:55.977558 13365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 07:01:55.977667 13365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 07:01:55.987375 13365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.40.20:42541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.40.62:43037,127.4.40.61:44453,127.4.40.60:44825
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/krb5kdc/kudu_127.4.40.60.keytab
--principal=kudu/127.4.40.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.40.60:44825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.40.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 05:57:29 UTC on ccycloud.quasar-vkgtjh.root.comops.site
build id 4579822
ASAN enabled
I20260811 07:01:55.988714 13365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 07:01:55.990028 13365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 07:01:56.001938 13377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:56.001878 13373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 07:01:56.001978 13372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 07:01:56.003636 13365 server_base.cc:1034] running on GCE node
I20260811 07:01:56.004480 13365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 07:01:56.005822 13365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 07:01:56.007081 13365 hybrid_clock.cc:648] HybridClock initialized: now 1786431716007024 us; error 48 us; skew 500 ppm
I20260811 07:01:56.007412 13365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431716, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 07:01:56.010795 13365 init.cc:377] Logged in from keytab as kudu/127.4.40.60@KRBTEST.COM (short username kudu)
I20260811 07:01:56.012547 13365 webserver.cc:460] Webserver started at http://127.4.40.60:34399/ using document root <none> and password file <none>
I20260811 07:01:56.013259 13365 fs_manager.cc:359] Metadata directory not provided
I20260811 07:01:56.013362 13365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 07:01:56.013628 13365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 07:01:56.016620 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data/instance:
uuid: "f8e941fb05e641fca4bc017ddf5d20e5"
format_stamp: "Formatted at 2026-08-11 07:01:56 on dist-test-slave-7vjv"
server_key: "2e9af43424e62d70d707d6dde42f67ac"
server_key_iv: "0c50d20e521c1c1d784ab4a396f996a6"
server_key_version: "encryptionkey@0"
I20260811 07:01:56.017354 13365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal/instance:
uuid: "f8e941fb05e641fca4bc017ddf5d20e5"
format_stamp: "Formatted at 2026-08-11 07:01:56 on dist-test-slave-7vjv"
server_key: "2e9af43424e62d70d707d6dde42f67ac"
server_key_iv: "0c50d20e521c1c1d784ab4a396f996a6"
server_key_version: "encryptionkey@0"
I20260811 07:01:56.022429 13365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 07:01:56.025741 13396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 07:01:56.027051 13365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 07:01:56.027230 13365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data,/tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal
uuid: "f8e941fb05e641fca4bc017ddf5d20e5"
format_stamp: "Formatted at 2026-08-11 07:01:56 on dist-test-slave-7vjv"
server_key: "2e9af43424e62d70d707d6dde42f67ac"
server_key_iv: "0c50d20e521c1c1d784ab4a396f996a6"
server_key_version: "encryptionkey@0"
I20260811 07:01:56.027458 13365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 07:01:56.039429 13365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 07:01:56.043967 13365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 07:01:56.044292 13365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 07:01:56.062295 13365 rpc_server.cc:307] RPC server started. Bound to: 127.4.40.60:44825
I20260811 07:01:56.062333 13491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.40.60:44825 every 8 connection(s)
I20260811 07:01:56.063633 13365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/data/info.pb
I20260811 07:01:56.067235 13495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 07:01:56.072367 4256 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu as pid 13365
I20260811 07:01:56.072577 4256 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCgIwdj/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786431333861527-4256-0/minicluster-data/master-2/wal/instance
I20260811 07:01:56.073019 4256 external_mini_cluster.cc:1428] Setting key 04b0de1e0ecc075afd2dfcf7ce054d86
I20260811 07:01:56.074383 13495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431716, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 07:01:56.098129 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.077191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.62:43037 (local address 127.0.0.1:49464)
0811 07:01:56.077932 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:56.077980 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:56.078286 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:56.078890 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:56.078897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:56.079520 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:56.079905 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.079922 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.081046 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.081050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.082208 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.082220 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.082983 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.082990 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.083184 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.083942 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:56.083962 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:56.086622 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:56.093652 (+ 7030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.093665 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.093678 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.094202 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.094616 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.094623 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.094628 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.094830 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.095515 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:56.095525 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:56.096001 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 07:01:56.096276 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 07:01:56.096716 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":82,"thread_start_us":326,"threads_started":1}
I20260811 07:01:56.099355 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.077242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49464 (local address 127.4.40.62:43037)
0811 07:01:56.078554 (+ 1312us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.078562 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.078583 (+ 21us) server_negotiation.cc:407] Connection header received
0811 07:01:56.078631 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.078634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.078689 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.078811 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.080084 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.080880 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.082370 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.082755 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.082889 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.086854 (+ 3965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.086862 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.086867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.086916 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.093505 (+ 6589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.094355 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.094362 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.094367 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.094502 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.094958 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.094965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.094970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.095314 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.095442 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.098589 (+ 3147us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.098834 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1189,"thread_start_us":1032,"threads_started":1}
I20260811 07:01:56.099923 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.074907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59650 (local address 127.4.40.60:44825)
0811 07:01:56.075780 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.075791 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.075830 (+ 39us) server_negotiation.cc:407] Connection header received
0811 07:01:56.078132 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.078167 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.078506 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.078706 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.080239 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.081382 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.082582 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.083083 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.083289 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.088373 (+ 5084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.088383 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.088398 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.088463 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.093360 (+ 4897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.094444 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.094457 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.094463 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.094721 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.095254 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.095262 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.095268 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.096123 (+ 855us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.096493 (+ 370us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.097605 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.097926 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":658,"thread_start_us":402,"threads_started":1}
I20260811 07:01:56.101804 13495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431716, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.62@KRBTEST.COM
I20260811 07:01:56.111724 13219 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 4
I20260811 07:01:56.116976 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.102620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.61:44453 (local address 127.0.0.1:38610)
0811 07:01:56.103276 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:56.103289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:56.103418 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:56.103756 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:56.103761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:56.104222 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:56.104365 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.104374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.105371 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.105376 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.106226 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.106237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.106935 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.106941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.107158 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.107768 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:56.107792 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:56.110125 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:56.114341 (+ 4216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.114351 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.114356 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.114901 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.115298 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.115305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.115311 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.115495 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.116032 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:56.116038 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:56.116218 (+ 180us) client_negotiation.cc:769] Sending connection context
0811 07:01:56.116360 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 07:01:56.116628 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571}
I20260811 07:01:56.117049 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.102813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38610 (local address 127.4.40.61:44453)
0811 07:01:56.103021 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.103029 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.103372 (+ 343us) server_negotiation.cc:407] Connection header received
0811 07:01:56.103522 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.103527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.103601 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.103733 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.104513 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.105217 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.106366 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.106793 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.106926 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.110325 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.110333 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.110339 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.110383 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.114164 (+ 3781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.115044 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.115053 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.115055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.115188 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.115610 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.115615 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.115619 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.115891 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.116058 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.116470 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.116680 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:01:56.118036 13495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } has no permanent_uuid. Determining permanent_uuid...
I20260811 07:01:56.120573 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.103316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49468 (local address 127.4.40.62:43037)
0811 07:01:56.103505 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.103510 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.103525 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:01:56.103587 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.103590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.103667 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.103777 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.105017 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.105670 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.106902 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.107454 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.107590 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.113138 (+ 5548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.113144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.113148 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.113185 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.117018 (+ 3833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.118017 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.118023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.118027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.118195 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.118633 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.118638 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.118641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.119008 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.119297 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.119899 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.120167 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.62@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431716, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.60@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 07:01:56.129194 13223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.112397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59652)
0811 07:01:56.112589 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:56.112609 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:56.112712 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:56.113352 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:56.113357 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:56.113876 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:56.114121 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.114140 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.115408 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.115416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.117018 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.117029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.118004 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.118012 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.118210 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.119046 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:56.119072 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:56.121914 (+ 2842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:56.126454 (+ 4540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.126459 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.126463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.126889 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.127336 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.127344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.127351 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.127525 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.128215 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:56.128220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:56.128382 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 07:01:56.128539 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 07:01:56.128849 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 07:01:56.130102 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.112858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59652 (local address 127.4.40.60:44825)
0811 07:01:56.113060 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.113066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.113084 (+ 18us) server_negotiation.cc:407] Connection header received
0811 07:01:56.113132 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.113136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.113206 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.113306 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.114290 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.115277 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.117200 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.117849 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.117973 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.122128 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.122136 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.122141 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.122186 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.126313 (+ 4127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.127013 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.127019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.127022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.127189 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.127661 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.127668 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.127674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.128038 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.128240 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.129492 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.129758 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431714, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.40.61@KRBTEST.COM
I20260811 07:01:56.134428 13503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.118713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59654)
0811 07:01:56.118862 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:56.118876 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:56.118982 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:56.120134 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:56.120139 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:56.120559 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:56.120741 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.120753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.122399 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.122405 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.123829 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.123847 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.124653 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.124661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.124927 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.125478 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:56.125498 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:56.127933 (+ 2435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:56.131956 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.131961 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.131965 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.132416 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.132782 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.132787 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.132791 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.132920 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.133442 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:56.133446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:56.133584 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 07:01:56.133754 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 07:01:56.134061 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 07:01:56.134584 13507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.119012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59654 (local address 127.4.40.60:44825)
0811 07:01:56.119799 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.119807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.119830 (+ 23us) server_negotiation.cc:407] Connection header received
0811 07:01:56.119873 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.119877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.119940 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.120865 (+ 925us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.121024 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.122222 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.123969 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.124490 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.124592 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.128175 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.128181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.128185 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.128222 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.131833 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.132539 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.132544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.132547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.132673 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.133038 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.133043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.133046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.133318 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.133459 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.133856 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.134189 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":145,"server-negotiator.queue_time_us":658,"thread_start_us":425,"threads_started":1}
I20260811 07:01:56.137919 13219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e: Bootstrap starting.
I20260811 07:01:56.139539 13495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: Bootstrap starting.
I20260811 07:01:56.141017 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.122568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38624 (local address 127.4.40.61:44453)
0811 07:01:56.122734 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.122741 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.122830 (+ 89us) server_negotiation.cc:407] Connection header received
0811 07:01:56.123078 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.123085 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.123153 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.123214 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.124883 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.125991 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.127719 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.128394 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.128523 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.133657 (+ 5134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.133663 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.133668 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.133706 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.137207 (+ 3501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.138057 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.138062 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.138065 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.138226 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.138711 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.138717 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.138719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.139043 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.139321 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.140483 (+ 1162us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.140695 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:01:56.142078 13219 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:56.142880 13495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:56.143507 13219 log.cc:824] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:56.144243 13495 log.cc:824] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:56.146189 4256 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 07:01:56.146992 13219 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e: No bootstrap required, opened a new log
I20260811 07:01:56.147258 13495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: No bootstrap required, opened a new log
I20260811 07:01:56.149299 13172 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "996a92e568ba4a07a45af565cdffc03e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49468
I20260811 07:01:56.154284 13495 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.154633 13495 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8e941fb05e641fca4bc017ddf5d20e5, State: Initialized, Role: FOLLOWER
I20260811 07:01:56.155059 13495 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.155912 13518 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.156363 13518 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.157050 13495 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:56.156819 13219 raft_consensus.cc:348] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.157159 13219 raft_consensus.cc:729] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 996a92e568ba4a07a45af565cdffc03e, State: Initialized, Role: FOLLOWER
I20260811 07:01:56.157624 13219 consensus_queue.cc:260] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.158804 13524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.159525 13524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.159883 13219 sys_catalog.cc:564] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:56.159988 13172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 07:01:56.160202 13172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:56.164781 13172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.169721 13440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "996a92e568ba4a07a45af565cdffc03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f8e941fb05e641fca4bc017ddf5d20e5"
I20260811 07:01:56.170333 13440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:56.176455 13440 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 996a92e568ba4a07a45af565cdffc03e in term 1.
I20260811 07:01:56.176831 13172 leader_election.cc:288] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [CANDIDATE]: Term 1 election: Requested vote from peers 5876f1d15f48458490070eb1e9a1f7b6 (127.4.40.61:44453), f8e941fb05e641fca4bc017ddf5d20e5 (127.4.40.60:44825)
I20260811 07:01:56.177873 13133 leader_election.cc:302] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 996a92e568ba4a07a45af565cdffc03e, f8e941fb05e641fca4bc017ddf5d20e5; no voters:
I20260811 07:01:56.178509 13524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 07:01:56.166491 13365 master_runner.cc:186] Error getting master registration for 127.4.40.62:43037: OK, instance_id { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" instance_seqno: 1786431714970033 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:01:56.182245 13365 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:01:56.188905 13524 raft_consensus.cc:686] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [term 1 LEADER]: Becoming Leader. State: Replica: 996a92e568ba4a07a45af565cdffc03e, State: Running, Role: LEADER
W20260811 07:01:56.192456 13547 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 07:01:56.192628 13547 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 07:01:56.192015 13524 consensus_queue.cc:237] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.199079 13553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 996a92e568ba4a07a45af565cdffc03e. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.199498 13553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:56.203594 13582 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260811 07:01:56.209373 13140 tablet.cc:2369] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 07:01:56.222662 13505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.195947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49472 (local address 127.4.40.62:43037)
0811 07:01:56.196151 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.196157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.204535 (+ 8378us) server_negotiation.cc:407] Connection header received
0811 07:01:56.205200 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.205208 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.205297 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.205376 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.207889 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.209027 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.210984 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.211764 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.211865 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.215670 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.215677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.215682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.215742 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.219056 (+ 3314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.219933 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.219938 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.219941 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.220157 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.220644 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.220648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.220651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.221019 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.221317 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.221985 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.222220 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 07:01:56.242789 13582 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 07:01:56.208959 13094 master_runner.cc:186] Error getting master registration for 127.4.40.61:44453: OK, instance_id { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" instance_seqno: 1786431715520356 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 07:01:56.243243 13094 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 07:01:56.243705 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.200026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38628 (local address 127.4.40.61:44453)
0811 07:01:56.200243 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.200252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.223595 (+ 23343us) server_negotiation.cc:407] Connection header received
0811 07:01:56.223848 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.223855 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.223945 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.224032 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.225941 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.227028 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.229300 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.229937 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.230042 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.234309 (+ 4267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.234319 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.234327 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.234371 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.238353 (+ 3982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.239712 (+ 1359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.239718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.239721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.239896 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.240687 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.240694 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.240701 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.241084 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.241337 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.242300 (+ 963us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.242709 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 07:01:56.246795 13353 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } attempt: 5
Aug 11 07:01:56 dist-test-slave-7vjv krb5kdc[13065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786431715, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.40.61@KRBTEST.COM for kudu/127.4.40.60@KRBTEST.COM
I20260811 07:01:56.259706 13507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.228047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59670 (local address 127.4.40.60:44825)
0811 07:01:56.228277 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.228283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.228758 (+ 475us) server_negotiation.cc:407] Connection header received
0811 07:01:56.229764 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.229770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.229860 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.229933 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.235290 (+ 5357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.236524 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.239703 (+ 3179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.240403 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.240516 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.249454 (+ 8938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.249460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.249474 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.249516 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.253442 (+ 3926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.255969 (+ 2527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.255974 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.255977 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.256192 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.257413 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.257419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.257424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.257744 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.258212 (+ 468us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.258798 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.259030 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 07:01:56.262727 13440 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [term 1 FOLLOWER]: Refusing update from remote peer 996a92e568ba4a07a45af565cdffc03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:56.264936 13553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:56.265460 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.247310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.40.60:44825 (local address 127.0.0.1:59674)
0811 07:01:56.247475 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 07:01:56.247494 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 07:01:56.247590 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 07:01:56.248227 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 07:01:56.248232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 07:01:56.248696 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 07:01:56.248883 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.248893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.250608 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.250615 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.252219 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 07:01:56.252236 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.253636 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.253641 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 07:01:56.253863 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.254457 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 07:01:56.254490 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 07:01:56.257110 (+ 2620us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 07:01:56.262580 (+ 5470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.262587 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.262594 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.263116 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.263596 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 07:01:56.263602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 07:01:56.263606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 07:01:56.263775 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 07:01:56.264514 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 07:01:56.264521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 07:01:56.264689 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 07:01:56.264818 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 07:01:56.265100 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 07:01:56.267521 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.247679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59674 (local address 127.4.40.60:44825)
0811 07:01:56.247918 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.247925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.247944 (+ 19us) server_negotiation.cc:407] Connection header received
0811 07:01:56.248001 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.248006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.248077 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.248203 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.249032 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.250132 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.252420 (+ 2288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.253208 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.253368 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.257276 (+ 3908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 07:01:56.257281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 07:01:56.257285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 07:01:56.257320 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 07:01:56.262295 (+ 4975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.263279 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.263284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.263287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.263443 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 07:01:56.263928 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 07:01:56.263932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 07:01:56.263934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 07:01:56.264355 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 07:01:56.266813 (+ 2458us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.266935 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.267145 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 07:01:56.277359 13353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: Bootstrap starting.
I20260811 07:01:56.286265 13353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: Neither blocks nor log segments found. Creating new log.
I20260811 07:01:56.288964 13353 log.cc:824] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: Log is configured to *not* fsync() on all Append() calls
I20260811 07:01:56.290982 13518 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 996a92e568ba4a07a45af565cdffc03e. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.291383 13518 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.311682 13518 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.312013 13518 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.313118 13524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.313793 13524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:56.316041 13582 catalog_manager.cc:1347] Generated new cluster ID: dc2b895be6d34e23a9867c906cf40444
I20260811 07:01:56.316169 13582 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 07:01:56.336037 13353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: No bootstrap required, opened a new log
I20260811 07:01:56.354739 13353 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.355201 13353 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5876f1d15f48458490070eb1e9a1f7b6, State: Initialized, Role: FOLLOWER
I20260811 07:01:56.355749 13353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } }
I20260811 07:01:56.360235 13582 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 07:01:56.361059 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.362052 13582 catalog_manager.cc:1504] Loading token signing keys...
I20260811 07:01:56.361999 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.362367 13353 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 07:01:56.362411 13312 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "996a92e568ba4a07a45af565cdffc03e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5876f1d15f48458490070eb1e9a1f7b6"
I20260811 07:01:56.362862 13312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 07:01:56.367240 13312 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 996a92e568ba4a07a45af565cdffc03e in term 1.
I20260811 07:01:56.368541 13310 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [term 1 FOLLOWER]: Refusing update from remote peer 996a92e568ba4a07a45af565cdffc03e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 07:01:56.377243 13553 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 07:01:56.381122 13582 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e: Generated new TSK 0
I20260811 07:01:56.382427 13582 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 07:01:56.458323 13614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.458838 13614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.459367 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 996a92e568ba4a07a45af565cdffc03e. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.459725 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 07:01:56.461356 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "996a92e568ba4a07a45af565cdffc03e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "996a92e568ba4a07a45af565cdffc03e" member_type: VOTER last_known_addr { host: "127.4.40.62" port: 43037 } } peers { permanent_uuid: "5876f1d15f48458490070eb1e9a1f7b6" member_type: VOTER last_known_addr { host: "127.4.40.61" port: 44453 } } peers { permanent_uuid: "f8e941fb05e641fca4bc017ddf5d20e5" member_type: VOTER last_known_addr { host: "127.4.40.60" port: 44825 } } }
I20260811 07:01:56.461844 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 996a92e568ba4a07a45af565cdffc03e [sys.catalog]: This master's current role is: LEADER
I20260811 07:01:56.487430 13626 catalog_manager.cc:1259] Loaded cluster ID: dc2b895be6d34e23a9867c906cf40444
I20260811 07:01:56.487619 13626 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: loading cluster ID for follower catalog manager: success
I20260811 07:01:56.491642 13626 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: acquiring CA information for follower catalog manager: success
I20260811 07:01:56.494210 13626 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5876f1d15f48458490070eb1e9a1f7b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 07:01:56.523643 4256 mini_kdc.cc:348] Time spent kdestroy: real 0.121s user 0.000s sys 0.036s
W20260811 07:01:56.587497 13505 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.560065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49474 (local address 127.4.40.62:43037)
0811 07:01:56.560350 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.560360 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.584902 (+ 24542us) server_negotiation.cc:407] Connection header received
0811 07:01:56.585247 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.585254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.585343 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.585451 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.587142 (+ 1691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49474: BlockingRecv error: recv got EOF from 127.0.0.1:49474 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 07:01:56.590698 13361 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.585120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38630 (local address 127.4.40.61:44453)
0811 07:01:56.585313 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.585320 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.588598 (+ 3278us) server_negotiation.cc:407] Connection header received
0811 07:01:56.588959 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.588968 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.589076 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.589167 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.590363 (+ 1196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38630: BlockingRecv error: recv got EOF from 127.0.0.1:38630 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 07:01:56.606633 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.596968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59676 (local address 127.4.40.60:44825)
0811 07:01:56.597141 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.597147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.604428 (+ 7281us) server_negotiation.cc:407] Connection header received
0811 07:01:56.604576 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.604582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.604668 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.604772 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.606353 (+ 1581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59676: BlockingRecv error: recv got EOF from 127.0.0.1:59676 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 07:01:56.682041 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.657984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38644 (local address 127.4.40.61:44453)
0811 07:01:56.658205 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.658212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.673784 (+ 15572us) server_negotiation.cc:407] Connection header received
0811 07:01:56.674242 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.674249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.674511 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.674668 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.675456 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.676695 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.678804 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.679497 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.679619 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.680447 (+ 828us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:56.681167 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:56.681291 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.681554 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.681671 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 07:01:56.746078 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.732509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59692 (local address 127.4.40.60:44825)
0811 07:01:56.732721 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.732728 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.744056 (+ 11328us) server_negotiation.cc:407] Connection header received
0811 07:01:56.744331 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.744338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.744509 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.744621 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.745816 (+ 1195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59692: BlockingRecv error: recv got EOF from 127.0.0.1:59692 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 07:01:56.792733 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.778073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38648 (local address 127.4.40.61:44453)
0811 07:01:56.778292 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.778298 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.785709 (+ 7411us) server_negotiation.cc:407] Connection header received
0811 07:01:56.786171 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.786177 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.786272 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.786336 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.786963 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.788068 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.790245 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.790909 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.791032 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.791704 (+ 672us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:56.791873 (+ 169us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:56.791951 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.792266 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.792397 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 07:01:56.835800 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.825883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59708 (local address 127.4.40.60:44825)
0811 07:01:56.826077 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.826083 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.833757 (+ 7674us) server_negotiation.cc:407] Connection header received
0811 07:01:56.834264 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.834270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.834366 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.834439 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.835551 (+ 1112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59708: BlockingRecv error: recv got EOF from 127.0.0.1:59708 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 07:01:56.847595 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.841777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38654 (local address 127.4.40.61:44453)
0811 07:01:56.842025 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.842034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.842556 (+ 522us) server_negotiation.cc:407] Connection header received
0811 07:01:56.842770 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.842776 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.842861 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.842962 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.843406 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.844373 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.845865 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.846298 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.846463 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.846853 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:56.846963 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:56.847061 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.847214 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.847297 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 07:01:56.855140 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.853436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59720 (local address 127.4.40.60:44825)
0811 07:01:56.853612 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.853618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.853867 (+ 249us) server_negotiation.cc:407] Connection header received
0811 07:01:56.853994 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.853999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.854091 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.854195 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.854900 (+ 705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59720: BlockingRecv error: recv got EOF from 127.0.0.1:59720 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:01:56.867560 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.862010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38668 (local address 127.4.40.61:44453)
0811 07:01:56.862293 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.862311 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.862604 (+ 293us) server_negotiation.cc:407] Connection header received
0811 07:01:56.862800 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.862805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.862894 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.863005 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.863444 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.864468 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.865776 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.866273 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.866431 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.866802 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:56.866906 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:56.867003 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.867152 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.867233 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
W20260811 07:01:56.874915 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.873101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59724 (local address 127.4.40.60:44825)
0811 07:01:56.873352 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.873358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.873602 (+ 244us) server_negotiation.cc:407] Connection header received
0811 07:01:56.873737 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.873764 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.873872 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.873975 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.874696 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59724: BlockingRecv error: recv got EOF from 127.0.0.1:59724 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
I20260811 07:01:56.891439 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.885722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38680 (local address 127.4.40.61:44453)
0811 07:01:56.885911 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.885918 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.886258 (+ 340us) server_negotiation.cc:407] Connection header received
0811 07:01:56.886395 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.886401 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.886494 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.886587 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.886927 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.888059 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.889422 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.890166 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.890313 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.890675 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:56.890811 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:56.890916 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.891031 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.891112 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 07:01:56.899469 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.897456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59734 (local address 127.4.40.60:44825)
0811 07:01:56.897652 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.897658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.898023 (+ 365us) server_negotiation.cc:407] Connection header received
0811 07:01:56.898147 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.898153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.898246 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.898365 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.899215 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59734: BlockingRecv error: recv got EOF from 127.0.0.1:59734 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 07:01:56.938565 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.932880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38694 (local address 127.4.40.61:44453)
0811 07:01:56.933115 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.933121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.933399 (+ 278us) server_negotiation.cc:407] Connection header received
0811 07:01:56.933637 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.933642 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.933730 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.933831 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.934355 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.935189 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.936772 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.937201 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.937387 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:56.937819 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:56.937934 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:56.938050 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:56.938203 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:01:56.938293 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
W20260811 07:01:56.960144 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:56.957936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.4.40.60:44825)
0811 07:01:56.958157 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.958164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.958520 (+ 356us) server_negotiation.cc:407] Connection header received
0811 07:01:56.958659 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.958665 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.958800 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.958925 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:56.959863 (+ 938us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59748: BlockingRecv error: recv got EOF from 127.0.0.1:59748 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 07:01:57.001824 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:56.995459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.4.40.61:44453)
0811 07:01:56.995686 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:56.995692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:56.995969 (+ 277us) server_negotiation.cc:407] Connection header received
0811 07:01:56.996251 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:56.996257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:56.996379 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:56.996488 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:56.996905 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:56.998109 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:56.999686 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.000502 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.000649 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.000993 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.001142 (+ 149us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.001243 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.001366 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.001450 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 07:01:57.010142 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:57.008246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.4.40.60:44825)
0811 07:01:57.008436 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.008442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.008793 (+ 351us) server_negotiation.cc:407] Connection header received
0811 07:01:57.008925 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.008930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.009031 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.009138 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:57.009901 (+ 763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59762: BlockingRecv error: recv got EOF from 127.0.0.1:59762 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 07:01:57.082806 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:57.077489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.4.40.61:44453)
0811 07:01:57.077709 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.077719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.078120 (+ 401us) server_negotiation.cc:407] Connection header received
0811 07:01:57.078355 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.078359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.078430 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.078535 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:57.078986 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.079754 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.081115 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.081533 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.081678 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.082093 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.082208 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.082305 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.082458 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.082541 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 07:01:57.090478 13501 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 07:01:57.088733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.4.40.60:44825)
0811 07:01:57.088955 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.088961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.089240 (+ 279us) server_negotiation.cc:407] Connection header received
0811 07:01:57.089350 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.089353 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.089422 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.089501 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 07:01:57.090261 (+ 760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59766: BlockingRecv error: recv got EOF from 127.0.0.1:59766 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 07:01:57.194823 13547 catalog_manager.cc:1259] Loaded cluster ID: dc2b895be6d34e23a9867c906cf40444
I20260811 07:01:57.194983 13547 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: loading cluster ID for follower catalog manager: success
I20260811 07:01:57.197804 13547 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: acquiring CA information for follower catalog manager: success
I20260811 07:01:57.199862 13547 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8e941fb05e641fca4bc017ddf5d20e5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 07:01:57.227489 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:57.222643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38726 (local address 127.4.40.61:44453)
0811 07:01:57.222892 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.222899 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.223101 (+ 202us) server_negotiation.cc:407] Connection header received
0811 07:01:57.223263 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.223267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.223353 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.223436 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:57.223845 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.224616 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.225941 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.226320 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.226412 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.226767 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.226884 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.226952 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.227097 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.227160 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 07:01:57.239176 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:57.233416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59780 (local address 127.4.40.60:44825)
0811 07:01:57.233668 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.233674 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.233981 (+ 307us) server_negotiation.cc:407] Connection header received
0811 07:01:57.234199 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.234203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.234359 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.234488 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:57.234906 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.235627 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.237016 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.237416 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.237552 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.238037 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.238546 (+ 509us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.238719 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.238819 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.238898 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 07:01:57.252552 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:57.245530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38734 (local address 127.4.40.61:44453)
0811 07:01:57.245751 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.245759 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.246334 (+ 575us) server_negotiation.cc:407] Connection header received
0811 07:01:57.246711 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.246717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.246845 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.246912 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:57.247681 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.248688 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.250602 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.251172 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.251299 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.251779 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.251914 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.252017 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.252209 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.252296 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 07:01:57.252717 13829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:57.244731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.4.40.62:43037)
0811 07:01:57.245926 (+ 1195us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.245930 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.245945 (+ 15us) server_negotiation.cc:407] Connection header received
0811 07:01:57.246034 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.246037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.246184 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.247044 (+ 860us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:57.247214 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.248327 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.250097 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.250505 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.250714 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.251300 (+ 586us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.251783 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.251895 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.252334 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.252417 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":984,"threads_started":1}
I20260811 07:01:57.254334 13501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 07:01:57.246505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59794 (local address 127.4.40.60:44825)
0811 07:01:57.246693 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 07:01:57.246700 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 07:01:57.247332 (+ 632us) server_negotiation.cc:407] Connection header received
0811 07:01:57.247509 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 07:01:57.247514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 07:01:57.247604 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 07:01:57.247690 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 07:01:57.248382 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.249421 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.251511 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 07:01:57.252193 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 07:01:57.252309 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 07:01:57.253463 (+ 1154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 07:01:57.253600 (+ 137us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 07:01:57.253684 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 07:01:57.253904 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 07:01:57.254008 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 07:01:57.257504 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13094
I20260811 07:01:57.258745 13210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:57.382211 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13094
I20260811 07:01:57.418154 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13233
I20260811 07:01:57.419313 13344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:57.526854 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13233
I20260811 07:01:57.559702 4256 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu pid 13365
I20260811 07:01:57.560981 13482 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 07:01:57.668522 4256 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCgIwdj/build/asan/bin/kudu with pid 13365
2026-08-11T07:01:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4888 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9790 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (383783 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST